TRANSVERSE NONLINEAR INSTABILITY OF SOLITARY WAVES 
FOR SOME HAMILTONIAN PDE'S 
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Abstract. We present a general result of transverse nonlinear instability of 1-d solitary 
waves for Hamiltonian PDE's for both periodic or localized transverse perturbations. 
Our main structural assumption is that the linear part of the Id model and the trans- 
verse perturbation "have the same sign". Our result applies to the generalized KP-I 
equation, the Nonlinear Schrodinger equation, the generalized Boussinesq system and 
the Zakharov-Kuznetsov equation and we hope that it may be useful in other contexts. 



Contents 

1. Introduction 2 

2. General framework and results 3 

2.1. The unperturbed model 3 

2.2. The transversally perturbed model 5 

2.3. The resolvent equation 6 

2.4. The nonlinear problem 8 

2.5. Statement of the abstract results 18 

3. A sufficient condition for the existence of an unstable mode [k 

4. Criteria for the existence of the Evans function and the bounded frequencies 
resolvent bounds 11 

4.1. Reduction to an ODE 11 

4.2. Asymptotic behavior and consistent splitting 12 

4.3. Existence of the Evans function 12 

4.4. Resovent estimates in the periodic case 13 

4.5. Resolvent estimates in the localized case 13 

4.6. Proof of Lemma [4T21 14 

4.7. Proof of Lemma 14.31 16 

5. Criterion for the existence of multipliers 19 

5.1. Proof of Corollary E2 19 

5.2. Proof of Lemma 15.11 IS 

6. Proof of Theorem [1] (periodic perturbations) [2C 



2 



FREDERIC ROUSSET AND NIKOLAY TZVETKOV 



6.1. Construction of a most unstable eigenmode 

6.2. Construction of an high order unstable approximate solution 

6.3. Proof of Proposition 16.51 

6.4. Nonlinear instability (end of the proof of Theorem [T]) 

7. Proof of Theorem [2] (localized perturbations) 

8. Examples 

8.1. The generalized KP-I equation 

8.2. The nonlinear Schrodinger equation 

8.3. The Boussinesq equation 

8.4. The Zakharov-Kuznetsov equation 



21 
22 
26 
28 
30 

3J 

35 

40 
i2 

i£ 

50 

52 
53 



8.5. KP-BBM 

8.6. Final remark 
References 



1. Introduction 



A lot of two-dimensional dispersive equations possess one-dimensional solitary waves 
which are stable when submitted to one-dimensional perturbations but which are desta- 
bilized when submitted to general two-dimensional perturbations. This phenomenon has 
been known for a long time in the physics literature. For example, by using the Lax pair 
structure of the KP-I equation, it was proven in [32] that the KdV solitary wave seen as a 
Id solution of the KP-I equation is unstable. For non-integrable equations, the general in- 
stability theory of solitary waves of [10] does not seem to apply since the 1-d solitary wave 
is not a constrained critical point of the Hamiltonian of the 2d equation. Nevertheless, 
in some cases, the linear instability can be proven by some simple bifurcation arguments, 
for example, the linear instability of the Id solitary wave of the 2d Nonlinear Schrodinger 
equation (NLS) can be proven by the Zakharov-Rubenchik bifurcation argument for small 
transverse frequencies. Consequently, it seems interesting to reduce the proof of nonlinear 
instability to the search for unstable eigenmode for the linearized equation by proving that 
linear instability implies nonlinear instability for a large class of equations. 

In [27] , we have shown that the method developed by Grenier [12] for the incompressible 
Euler equation can be adapted to prove transverse instability of solitary waves in disper- 
sive models. More precisely, we have proven two nonlinear instability results for solitary 
waves of the Korteweg- de Vries and the Id Nonlinear Schrodinger equations (NLS), seen 
as solutions of the KP-I or the 2d NLS equations respectively and subject to periodic 
transverse perturbations. The linear instability in both cases was known. More precisely, 
in the KP-I case one has a complete understanding of the possible unstable modes for any 
fixed transverse frequency while in the NLS case unstable modes where detected thanks 
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to the Zakharov-Rubenchik bifurcation argument for small transverse frequencies. The 
possibility of describing all unstable modes in the KP-I case seems to be related to the 
Lax pairs structure of the KP-I equation (sometimes called complete integrability) . The 
Zakharov-Rubenchik bifurcation argument is a more general feature but does not seem 
to apply in some important cases such as the gKP-I equation, a case which is in the 
scope of the applicability of the present paper. Our goal here is to present a general trans- 
verse nonlinear instability theory of solitary waves, assuming the spectral instability of the 
solitary wave, for Hamiltonian PDE's obeying to some structural assumptions described 
below, the main one being that, in some sense, the transverse perturbation and the Id 
dispersion operator should have the same sign. More precisely, we state two instability 
results, one for transverse periodic boundary condition and one where the transverse di- 
rection is unbounded and the perturbations are localized. This last case was not studied 
in our previous work [27] and requires more work in the study of low frequencies. We 
also present a criterion to detect unstable modes, and thus to prove linear instability, in- 
spired by the work of Groves-Haragus-Sun [13], which is different and more flexible than 
the one presented in our previous work [27J for NLS. Finally, we check that our general 
theory can be applied to prove the linear and nonlinear instability of Id solitary waves 
in the generalized KP-I equation, the 2d NLS equation, a Boussinesq type equation, the 
Zakharov-Kuznetsov equation and the KP-BBM equation. 

Our method mainly depends on the Hamiltonian structure of the equation and we 
hope that the ideas of this paper may be extended to more general, not necessarily linear 
transverse perturbations. In particular, we hope that our approach may be useful to get 
transverse instability for some more complicated fluid mechanics models. 

The paper is organized as follows. We first describe the general framework and our as- 
sumptions. Then we state two abstract instability results under the additional assumption 
of the existence of an unstable mode of the linearized equation. Some of our assumptions 
will be easily verified in the applications. Other assumptions such as the existence of 
multipliers or the bounded frequencies resolvent estimates are not a general feature in the 
considered framework. For that reason in the later sections we present criteria insuring 
the validity of these assumptions and in particular, a criterion for the existence of unstable 
eigenmodes. These criteria will be usefull to analyze our concrete examples. In the last 
section of the paper, we apply the general theory to various examples. 

Acknowledgement. We are indebted to Jean-Claude Saut for several discussions on 
the subject and in particular for providing us the reference |13j . 

2. General framework and results 

2.1. The unperturbed model. For s a real number, we consider the Sobolev spaces 
H s = ii~ s (]R; where d > 1 is an integer and we denote its norm by | • \ s . The I? norm 
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will be simply denoted by | • | and the L 2 scalar product by (•,■)• We consider the equation 

(2.1) 8 t u = J(L u + VF(u)), 

where F € C°°(IR d ;IR), F(0) = and the linear operators J and L are such that : 

• J is a Fourier multiplier which is skew-symmetric for the I? scalar product with 
domain containing H 1 (thus J is of order at most one) and such that Ker J = {0}. 

• Lq is a Fourier multiplier which is a symmetric operator with a self adjoint reali- 
sation on L 2 (]R;]R' : ') with domain D(Lq) containing H 2 . Moreover, Lq is coercive, 

(2.2) C~ x \u\l < (L u,u) < C\u\\ . 

Note that since J and Lq are Fourier multipliers, they commute with the x derivative and 
hence we have that J e B(H S , H 13 " 1 ) and Lq G B(H s , H s ~ 2 ) for every s. 
Equation (|2.ip can thus be written in the Hamiltonian form 

1 f°° 
d t u = JVH(u), H(u) = -(L u,u)+ F(u)dx. 

^ J-oo 

One may imagine situations when J and Lq are of higher orders. In these cases some 
modifications of the considered framework should be done. However, in all our examples 
Lq is of order 2 and J of order or 1. 

We are interested in the stability of stationary solutions of (|2,1|) . Since J is into, they 
are critical points of the Hamiltonian H, i.e. we have VH(Q) = LqQ + VF(Q) = 0. We 
focus on the case where Q is smooth, Q G H°°. Next, we consider the linear operator 
associated to the second variation of the Hamiltonian at Q : 

L = D U (VH)(Q) =L + R, Ru = V 2 F{Q)u. 

Note that R is a bounded operator on H s for every s > since Q and F are smooth. 
Consequently, L is a self adjoint operator on L 2 with domain D(Lq). Our main assumption 
on L is that its spectrum is under the form 

(2.3) a(L) = {fi} U {0} U S, 

where [i < is a simple eigenvalue, is an eigenvalue of finite multiplicity and E C [a, +oo) 
for some a > 0. Moreover, the eigenspaces associated to /i and zero are made of smooth 
eigenvectors (i.e which are in H°°). Many of our arguments remain valid if a(L)D] — oo, 0] 
contains a finite number of eigenvalues of finite multiplicities. We will be interested in 
situations where Q is a stable object for (|2. 1 j) . Note that the spectral assumption (|2.3[) 
is one of the main assumption which allows to prove the stability of Q by the Grillakis- 
Shatah-Strauss method [TO] . 
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2.2. The transversally perturbed model. We are interested in the stability of Q when 
(|2.1|) can be embedded in a larger Hamiltonian equation 

(2.4) d t u = J(d y )(L u + VF(u) + S(d y )u), 

where now u also depends on y with y G T a =M/27raZ or y G K. and Lq acts in a natural 
way on functions of 2 variables. The operators J(d y ), S(d y ) are operator valued Fourier 
multipliers in y, i.e. if T y stands for the Fourier transform in y, we have 

Fy(S(dy) U )(k) = S(ik)Fy(u)(k), Ty (J \3y) U) (k) = J \ik)T y (u) (k) . 

Moreover, S(ik) and J(ik) are now Fourier multipliers in x. In the following, we still 
denote by (•,•) and | • | s the complex scalar product of L 2 (R, C d ) and the H s norm for 
complex valued functions respectively. 

2.2.1. Assumptions on the operator J(ik). For every k, J(ik) is a Fourier multiplier such 
that: 

• J(ik) and J(ik)Lo are skew symmetric on L 2 (M), J(0) = J, 

• the domain of J(ik) contains H , Ker J(ik) = {0} and we have the uniform bound 

(2.5) 3C>0, Vfc, \J{ik)u\ < C\u\i, Vn G H 1 , 

• The commutator [R, J(ik)] is a uniformly bounded operator on I? : 

(2.6) 3C>0, Vfc, \([R,J(ik)]w,w)\<C\w\ 2 . 

Note that since J(0) = J, J{d y )u = Ju if « depends only on x. We also point out that 
the assumption (|2,6p is obviously verified when J is a bounded operator on L 2 . 

2.2.2. Assumptions on the operator S(ik). For every k, S(ik) is a Fourier multiplier such 
that: 

• S(ik) is non-negative and symmetric, J(ik)S(ik) is skew symmetric, S(0) = 0, 

• S(ik) has a self-adjoint realisation on L 2 with domain 2?g independent of k for 

• J{ik)S{ik)J{ik) and J{ik)S{ik)d x belong to £(i/ 2 (M), L 2 (M)), 

• Let us set = (w,S(ik)w), then there exists a non-negative continuous 
function (possibly unbounded) C(k) such that 

(2.7) | J(ifc)5(ifc)n| L2 < C(k)\u\ sm , V n G P 5 ■ 

Note that (|2.4p also has an Hamiltonian structure with Hamiltonian given by 

H(u) = J (^(L Q u,u) + -(S(d y )u,u) + J F(u)dx^ dy. 
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Moreover, we also point out that Q is still a stationary solution of (12. 4p and more generally 
that if u is a (reasonable) solution of (|2.4p which does not depend on y, then u actually 
solves (ETCi 

2.2.3. Compatibility between S{ik) and L. We assume that there exists K and cq > such 
that for every \k\ > K, 

(2.8) (Lv, v) + (S(ik)v, v) > co\v\f, Vu£ if 2 n P s . 

This is one of our main structural assumption which roughly says that S and Lq have the 
same sign. This assumption is valid for example for the KP-I equation and the 2d NLS 
equation but not for the KP-II equation or the hyperbolic Schrodinger equation. 

2.3. The resolvent equation. In this subsection, we state our assumptions on the lin- 
earization of (|2.4j) about Q. Since S(d y ) is a linear map, the linearization of (|2.4j) about 
Q reads 

(2.9) v t = J{d y ){L + S{dy))v. 

Definition 2.1. An unstable mode for {2.1$ is a function U E 1? n T>(S(ik)) such that 
for some a E C with Re{a) > and some k E R, t/ie problem \2. 9\) has a solution of the 
form 

(2.10) u(t,z,y) = e at U(x)e iky . 

We call a the amplification parameter and k the transverse frequency associated to U. 
Thus if U is an unstable mode then it is a solution of the eigenvalue problem 

(2.11) aU = J{ik){L + S(ik))U, U E L 2 {R; C d ). 

2.3.1. Assumption of existence of an Evans function and Id stability. We assume that 
there exists a function D(o~,k) (Evans function) such that for every k, D(-,k) is analytic 
in Re a > and such that there exists an unstable mode (I2.10P if and only if D(a, k) = 0. 
We also assume that all the possible unstable eigenmodes are smooth (H°°) and that Q 
is spectrally stable with respect to one-dimensional perturbations which reads: 

(2.12) D(a, 0)^0, Re<r>0. 

A concrete criterion for the existence of the Evans function will be given in section [H In 
most examples we have in mind, (|2.1ip can be reduced to an ordinary differential equation 
and hence, as usual, the Evans function will be defined as a Wronskian determinant 
associated to an ODE obtained after some manipulations from (|2.1ip . 
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Next, let us consider the resolvent equation for Re(a) > 

(2.13) aU = J{ik)LU + J{ik)S{ik)U + J{ik)F, 

U G H°°(R; C d ) n V{S{ik), F G H°°(R, C d ) . 

2.3.2. Bounded frequencies resolvent bounds in the periodic case. We assume that there 
exists q such that for every k and every /C compact set in Re a > 0, every s > 0, there 
exists Ck^K,,s such that if D{-, k) does not vanish on /C, then for every F G there 
is a unique' solution £/ G H°°(R) C\V(S(ik)) of ([27L31) which satisfies 

(2.14) \u\ 8 < C k , K , s \F\ s+q , Va G JC. 

2.3.3. Bounded frequencies resolvent bounds in the localized case. When k is a continuous 
variable, we need some uniform dependence in k in the regime k ~ 0. We shall assume 
that the Evans function D is analytic in (a, k) for Re a > and 7^ and that there 
exists an analytic continuation D(a, k) which is analytic in {Re a > 0} x M. Moreover, we 
assume a strong ID stability 

(2.15) D(a, 0) + 0, Vcr, Re a > 

and the uniform (also with respect to k) resolvent bound : there exists q > such that 
for every compact set K, in {Re a > 0} and M > 0, there exists Ck,,m,s such that if 
D(a, k) does not vanish on IC x (0, M], then for every F G if 00 , there is a unique solution 
U G n V(S(ik)) of (l2~13ll which satisfies 

(2.16) M s < CjcM.sli^W V(<7,fc)e£x (0,M]. 

As we shall see below, in most examples the existence of the Evans function and the 
bounds (|2.14p . (|2.16p can be obtained by ODE techniques. We shall give below a simple 
criterion which allows to obtain (|2.14p . (|2.16p . We also point out that we allow the case 
where D(a, 0) is different from D(a, 0) since we have not assumed continuity of D at k = 0. 
Typically D(a, k) is the determinant of a matrix of fixed size for k 7^ and D(o~, 0) is the 
determinant of a smaller matrix. 

As we shall prove, the assumptions (|2.3p . (|2.8p and the structural properties of the 
operators given in sections 12.2.11 12.2.21 are sufficient to ensure nice resolvent bounds in 
the energy norm H 1 for large |Im a\. The following assumption will be used to get the 
estimates of higher order derivatives. 
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2.3.4. Existence of a multiplier. We suppose that for every s > 2, there exists a self-adjoint 
operator M s such that there exists C > with 

(2.17) \(M s u,v)\ <C\u\ s \v\ s , (M s u,u) > \u\ 2 s - C\u\ 2 s ^ 
and 

(2.18) Re(J(ik)(L + S(ik))u, M s u) < C k \u\ s |u| s _i. 

The assumption (|2.18p will play a key role for the control on higher derivatives in a 
resolvent analysis below. In the cases of "semi-linear" problems we will be able simply to 
choose M s = d^~ 1 Ld^~ 1 . 

2.4. The nonlinear problem. Finally, we make a set of assumptions on the nonlinear 
problem (|2.4p . Denote by HP the Sobolev type spaces on K x 1 or 1 x T„ with the norms 
|| • || s . We denote by || • || the norm of I? . Consider the problem 

(2.19) 8 t u = J(d y )(L u + VF(u a + u) - VF(u a ) + S{d y )u) + J(d y )G, u(0) = 0, 

where u a is a smooth function bounded with all its derivatives and G £ C(R; HP) for every 
s. We suppose that the problem (|2.19p is locally well-posed in the sense that for every 
u a and G satisfying the previous assumptions there exists a time T > and a solution 
of (I2.19|) in C([0, T]; H s ) for every s > so (sq > being sufficiently large), unique in a 
suitable class. Finally, we assume the that tame estimate 

(2.20) | ld^J(d y ) {DVF(w + v)-v), ^Qf «)) | < u(\\w\\ ws+ t^ + ||t,||.) ||t;||2 

holds for every a, (3, a + (3 = s, where u is a continuous non-decreasing function with 
ui(0) = and ((•, •)) is the L 2 scalar product for functions of two variables. 

This last assumption together with the properties of the operators J and Lq will ensure 
the existence of an HP energy estimate for (12.19|) . 



2.5. Statement of the abstract results. Let us state our first instability result for ([27 
with R x T a as a spatial domain. 

Theorem 1 (Nonlinear transverse periodic instability). Consider the Hamiltonian equa- 
tion \2.1$ and suppose that the assumptions of the previous sections hold true, except the 
assumptions of Sections \2.3.<& Assume also that there exists an unstable mode with corre- 
sponding transverse frequency fco / 0. Then we have nonlinear instability of (12. 4p defined 
onlx T 27r /fc . More precisely for every s > 0, there exists 77 > such that for every 5 > 0, 
there exists Uq S H°°(R x T 27r /fc ) and a time T s ~ | log 5\ such that \\uq — Q\\ s < S and the 
solution u s of (|2.4p with data Uq remains in HP on [0, T s ] and satisfies d{u & {T & ),T) > r) 
where T is the space of L 2 (R) functions depending only on x and d{u,!F) = inf„ e jr v\\. 
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Notice that we have a strong instability statement since we measure the initial pertur- 
bation in a strong norm such as || • || s while the instability occurs in the weaker norm I? . 
Our second result concerns fully localized perturbations. 

Theorem 2 (Nonlinear transverse localized instability). Consider the Hamiltonian equa- 
tion \2.1$ and suppose that the assumptions of the previous sections hold true. Assume 
also that there exists an unstable mode with k ^ 0. Then we have nonlinear instability of 
(12,41) posed on R 2 . More precisely for every s > 0, there exists r\ > such that for every 
5 > 0, there exists Uq and a time T 5 ~ | log5| such that \\uq — Q\\ s < 5 and the solution 
u s of ([231) with data u 5 remains defined on [0,T 5 ], i.e. u s - Q £ IF, W G [0, T 5 ] and 
satisfies d(u s (T s ), J 7 ) > r\ where again T is the space o/L 2 (M) functions depending only 
on x and d{u,!F) = inf^gjF \\u — v\\. 

These theorems state that the existence of an unstable eigenmode implies nonlinear 
orbital instability of the solitary wave. Indeed, the orbit of Q under the action of all the 
possible groups of invariance of (|2.ip remain in T . In particular our results exclude the 
possibility of orbital stability of Q with respect to the spatial translations. More precisely 
our result implies that 

inf \\ u {T s )-Q{.-a)\\ > V . 

There are many assumptions in these theorems, nevertheless, some of them will be 
very easy to check on examples, for example the structural assumption 12.2. Ij 12.2.21 The 
ones which are more difficult to check are the assumptions 12.3. H 12.3.21 12.3.31 12.3.41 that 
is to say, the existence of an Evans function and of multipliers, the bounded frequencies 
resolvent bounds, and also the assumption on the existence of an unstable eigenmode. 
Consequently, the next sections are devoted to the proof of more concrete criteria which 
ensure that these assumptions are verified and which are easy to test on examples. 

Let us explain the main steps in the proof of Theorem 2. The inspiration comes from 
the work of Grenier [12] in fluid mechanics problems. We believe that this scheme is quite 
general and may be useful in other contexts. 

1. The first step is to prove that the possible unstable modes in the sense of Def- 
inition 2.1 above necessarily belong to a compact set both with respect to the 
transverse frequency and the amplification parameter. This allows to find the 
most unstable mode i.e. with the largest real part of the amplification parame- 
ter (note that there exists at least an unstable eigenmode by assumption) and to 
define a first approximate growing solution by a wave packet construction in the 
framework of Theorem [2j 

2. The second step is to evaluate, both from above and below, in a suitable norm 
(here it is L 2 ) the first approximate growing solution given by step 1. In the proof 
of Theorem [21 we need to use the Laplace method and some properties of the curve 
k -» a{k). 
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3. The third step is, following Grenier [12j . the construction of a refined approximate 
solutions which is carefully estimated from above. Since we deal with Hamil- 
tonian PDE's this step requires a different argument compared to similar esti- 
mates for diffusive problems. Here we reduce the matters to resolvent bounds for 
a — J{ik){L + S(ik)) for c's with real parts larger that the amplification parameter 
of the most unstable mode and any k in the (compact) set of possible transverse 
frequencies. 

4. The last step is to estimate the difference between the refined approximate solution 
and the true solution on the interval [0, T 5 ] by energy estimates. The analysis in 
this step is quite flexible and seems to apply each time we have H s energy estimates 
for the full 2d problem. 

The paper is organized as follows. In section [3] we give a criterion for the existence of an 
unstable eigenmode, in section we give criteria for the existence of the Evans function 
and the bounded frequencies resolvent bounds and in section [5j we give a criterion for the 
existence of multipliers satisfying (I2.17p . (|2.18p . The two next sections are devoted to the 
proof of Theorem Q] and [2] and finally, the last section is devoted to the study of various 
examples for which we check that the general theory can be applied. 

3. A SUFFICIENT CONDITION FOR THE EXISTENCE OF AN UNSTABLE MODE 

In this section, we give a simple criterion which ensures the existence of an unstable 
eigenmode. This criterion is inspired by the work [13]. Consider the symmetric operator 



Since J(ik)S(ik)J(ik) G B(H 2 ,L 2 ) by assumption I2.2.H we get that the domain T> of 
Mfe contains H 4 (indeed, J(ik) is at most a first order operator and L is a second order 
operator). 

A simple criterion for the existence of an unstable eigenmode is given by the following 
statement. 

Lemma 3.1. Assume that for every k and every u real-valued J(ik)u and S(ik)u are also 
real valued. Next, assume that there exists ko ^ such that zero is a simple eigenvalue 
of Mfc with corresponding real-valued nontrivial eigenvalue <p S H°° normalized so that 
IMIl 2 (R) = 1- Finally, assume that is a Fredholm map of index zero, that depends 
smoothly on k for k close to ko and the non degeneracy condition 



Then there exists k in a vicinity of ko and a > such that there exists an unstable mode 
with amplification parameter a and transverse frequency k. 



defined by 



M k = J{ik)LJ{ik) + J(ik)S(ik)J(ik). 



(3.1) 
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As we shall see, this criterion can be used on many examples. 

Proof of Lemma \3.1l We need to solve the problem 

av = J(ik)Lv + J(ik)S{ik)v, v G L 2 (R) 

for k close to ko and a close to 0. We shall seek for a real and v real- valued. This is 
legitimate since by assumption J{ik)v and S{ik)v are real-valued if v is real-valued. We 
shall look for k = k(a) with k(0) = ko. Since Ker J(ik) = {0}, we look for v under the 
form v = J(ik)u, u G L? . Therefore, we need to solve the problem F(u, k, a) = 0, where 

F(u, k, a) = Mfc(u) — aJ(ik)u . 

We search for u of the form u = (p + w with w£V = {u£VP\ L 2 (R, R d ) : (n, ip) = 0}. 



since <p is an eigenvector of Mk Q by assumption. Next for (w, /i) £ P x 1, we have 



Thanks to (pTT]) the linear map D W)k G(0, k , 0) is a bijection from Pxlto L 2 (M,M d ). 
Consequently, by the implicit function theorem, for cr close to there exist w(a) G P and 
/c(<t) G R with w;(0) = and fe(0) = ko such that G(w(a), k(a),a) = 0. This completes the 



4. Criteria for the existence of the Evans function and the bounded 

frequencies resolvent bounds 

In this section we describe some concrete criteria in order to ensure the assumptions of 
sections 12.3.11 12.3.21 and 12.3.31 The first assumption roughly says that we can reduce the 
eigenvalue problem (|2.1ip to an ordinary differential equation. 

4.1. Reduction to an ODE. We thus assume that there exists a Fourier multiplier 
R(a, k) such that R(a, k) G B{H s+lk ,H s )) for every s > and that Ker R = {0}. More- 
over, we assume the block structure 




D Wjk G(0,ko,0)[w,(i] = M ko w + n([—M k ] k=ko ip) . 



proof of Lemma 13.11 



□ 



(4.1) 




Pi (c, k) 
P 2 (a,k) 




where : 
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• For every k, P\{a,k) is a r x r matrix of differential operators of order > 1 
with coefficients which depend analytically on a, 

(4.2) Pi(<J,k) =d™Hd + --- , 

• for every k, P2(o~, k) is an operator of order < m^ — 1 i.e. P2(cr, k) G B(H s+rrik ~ 1 , H s ) 
for every s > 

• For every k, E(a,k) is invertible and E{a, k) -1 £ B(H S , H s ) for every s > 0. 

• There exists (/, m) such that for every fe / 0, (Ik, mk) = (I, m) and Iq < I, tuq < m. 
Moreover, all the operators depend continuously on a for Re a > for each fixed k. 

Because of the triangular block structure (|4.ip . the study of the resolvent equation 
(|2.13p can be reduced to the study of the ordinary differential equation 

(4.3) Pi (a, k)m = (R(a, k)J{ik)F) x 

by using the block decomposition U = {u\,U2) t £ C r x C d ~ r . Note that we allow the 
possibility that r = d, which means that the resolvent equation can be directly reduced to 
an ordinary differential equation by applying the operator R(cr, k). 
We can rewrite fj4.3|) as a first order ordinary differential equation 

(4.4) ^ = A(x,a,k)V + ¥, 

where A(x,a,k) E A^jvju(C), iV& = r is a matrix which depends smoothly on x, ana- 
lytically on a and 

(4.5) F =((),••• ,0,(R(a,k)J(ik)F) 1 ). 

Note that A(x, a, k) is in general not "continuous" at k = 0, since for k = 0, the dimension 
of the matrix may be different. 

With our reduction assumptions, we have unstable eigenmodes if and only if the ODE 
(|4,4|) with F = has a nontrivial L 2 solution. 

4.2. Asymptotic behavior and consistent splitting. We add the assumption that 
there exist A^ia, k) and C > 0, a > 0, such that for every x,k and every a, 

(4.6) \A(x, a, k) - A^ia, k)\ < Ce~ a ^, 

and that the spectrum of A oa (a, k) does not meet the imaginary axis for Re(a) > 0. 



4.3. Existence of the Evans function. 

Lemma 4.1. Under the assumptions of sections \4-l\ \4-$\ there exists a function D(o~,k) 
(Evans function) which is analytic in Re(a) > 0, for every k and such that D(o~, k) = if 
and only if there exists a non trivial eigenmode solution of (12. lip . 
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Proof. By classical arguments (see e.g. [T]), the assumptions of section B~2l allows to define 
an Evans function D(o~,k) for (|4.4ft which is an analytic function in Re(a) > 0, for every 
k and such that D(a, k) = if and only if there exists a non trivial L 2 (M; W 7lk r ) solution 
of V = A(x,cr, k)V which is actually exponentially decreasing. Thanks to the reduction 
assumptions 14. II above, this is equivalent to the existence of a nontrivial solution of (|2.1ip . 

4.4. Resovent estimates in the periodic case. Under the above assumptions, we can 
prove : 

Lemma 4.2. Let R{a,k) satisfying assumptions (|4.1j) and (14. 2ft . then, there exists q>0 
such that for every k, every s > and every compact JC C {Re a > 0}, there exists C^ ^s 
such that if D(-,k) does not vanish on K, then there is a unique solution U G H°° PiT>s 
of (|2.13p for every F G H°° which satisfies 



In other words, if one can prove the existence of R(o~, k) then one get the resolvent 
bounds (|2.14p on every compact which does not contain unstable eigenmode. 

4.5. Resolvent estimates in the localized case. To get (|2.14p in the localized case, 
we need some assumptions on the dependence of the various objects with respect to k. 
We assume that: 

i) R(a,k), P2(a,k), E(a,k) depend continuously on (a,k) for k / 0, Re a > and 
have continuous extensions up to {Re a > 0} x K. 

ii) Pi and thus A and A^ are analytic for k ^ 0, Re a > and have analytic 
extensions up to {Re a > 0} x K. 

Next, since the spectrum of ^00(0", k) does not meet the imaginary axis for Re(<r) > and 
k 7^ 0, we can define a projection on the stable subspace of Aoo which is analytic in a and 
k by the Dunford integral 



where T is a contour which encloses all the negative real part eigenvalues of A^, the 
projection on the unstable subspace is then given by Id— P^. Note that we had assumed 
that ^00(17,0+) = lim^^Q A QO (a, k) exist but we allow the presence of eigenvalues on the 
imaginary axis. We nevertheless assume: 

iii) the projection Poo((T, k) can be continued analytically to {Re(a) > 0} x R, 
This implies thanks to the Gap Lemma ([H]), p2]) that the Evans function can also be 
continued analytically to {Re(a) > 0} x K. The continuation of the function will be 
denoted by D(a,k). Recall that D(a, 0) may be different from D(a, 0). Indeed A(x,cr,k) 
is not continuous at zero and hence A(x,a,0) 7^ linu^o A(x, a, k). By construction, the 
same difference holds for the Evans function. 



(4.7) 



u\ s < Ck,K,sW 



s+q- 
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Finally, we also assume : 

iv) for every compact set K of {Re a > 0}, and every s > 0, there exists C > such 
that for every eigenvalue fi(a, k) of -Aoo(<7, k) 

(4.8) \\R(a,k) - R(a,0+)\\ BiHs+ i iHs) + \\J(ik) - J(0)\\b(h*+\h°) 
+\\R(a,k)J(ik)S(ik)\\ B{H s +mtHs) < Cp(k,K) 

where 

p(k,K)= inf |Re/z(<7,fc)| 

o-GK,/i(cr,fc)GSpA 00 (<T,fc) 

for every A; in a small disk D(0,r)\{0} and a £ K, where I, m and k are defined 
in section I4TT1 

Note that since S(0) = 0, this assumption is nontrivial only when there exists an eigenvalue 
of ^00(0", k such that Re/i(<7, k) vanishes at k = 0. 
Then, we can prove the following statement. 

Lemma 4.3. Assuming the existence of R(o~, k) given by assumptions \4-l\ an d as- 
sumptions i)-iv) above, then there exists q > such that for every s > 0, every compact 
K, C {Re a > 0} and M > 0, there exists Cic,m, s such that if D does not vanish on 
K, x [0, M] and D does not vanish on K,, then, for every F G H°° , there is a unique 
solution U G H°° n V{S{ik)) of (^33]) which satisfies 

(4.9) \u\ a < C;c,M,s\F\ s+q , Vo G K, Vk G (0, M\. 
Consequently, we have given criteria which allow to obtain (|2,16p 

4.6. Proof of Lemma 14.21 By using R(o~, k) and setting w = (u\, 112) *, we can rewrite 

aw = J(ik)(Lw + S(ij)w) + J(ik)F 

as 

(4.10) V x = A{a,k,x)V + M, 
and 

u 2 = -E(a, k)- 1 P 2 (a, k)m + E(a, k)' 1 (R(a, k)J{ik)F) 2 

with V(x) = ,d^-^ Ul (x)) andH= (0, • • • ,(R{a,k)J(ik)F) 1 ). 

The properties of E, J(ik) and Pi and the triangular structure already give 

\u2\s < C a (|tti| a+mfe _i + \F\i k+s+ i). 

Consequently, it suffices to prove that for every s > 0, 

\V\ S <C S \M\ S 

where V is the solution of the ODE (|4.10|) to get the result. 
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Let us denote by T(<r, k, x, x') the fundamental solution of V x = AV i.e. the solution such 
that T(a,k,x' ,x') = Id. Thanks to our assumption (14. 6j) on the behavior as \x\ — > oo of 
A(a, k, x), we can use classical perturbative ODE arguments (more precisely the roughness 
of exponential dichotomy, see [7] for example). Namely, the equation V x = AV has an 
exponential dichotomy on M + and R_, i.e., there exists projections P + (cr, k, x), P~(cr, k, x) 
which are smooth in the parameter a with the invariance property 

(4.11) T(cr, k, x, x')P ± (a, k, x') = P ± (a, k, x)T(a, k, x, x') 

and such that there exists C and a > such that for every U S C fc , and a £ /C, we have 

\T(a,k,x,x')P + (a,k,x')U\ < C e -<*(*-*') \P + (a,k,x')U\, x>x'>0, 

\T(a,k,x,x')(I - P + {a,k,x'))U\ < Ce a( - X ~^ \(I - P + (a,k,x'))U\, < x < x' , 

\T(a,k,x,x')p-{a,k,x,x')U\ < Ce a{x ~ xl) \P~(a, k, x')U\, x < x' < 0, 

\T(a,k,x,x')(I - p-(a,k,x'))U\ < Ce'^"^ \{I - P~(a, k,x'))U\, 0>x>x'. 

In particular, note that a solution T(cx, k, x, 0)V° is decaying when x tend to ±oo if and 
only if V° belongs to TZ{P ± (a, k, 0)). Since when a is in /C, the Evans function does not 
vanish, we have by definition no non trivial solution decaying in both sides and hence we 
have 

(4.12) K(P + (a,k,0)) nK(P-(a,k,0)) = {0}. 

Let us choose bases (r^ 1 , • • • , r N ±) oilZ{P ± {a, k, 0)) (where N + +N~ = N^) which depends 
on a in a smooth way (see |18| for example) then we can define 

M(a,k) = (r+,-- - ,r+ + ,r 1 ",--- ,r~_) 

and we note that M(a,k) is invertible for a G /C because of (|4.12p . With, these new 
notations, we note in passing that the Evans function can actually be defined by 

D(a,k) = detM(<7,fc). 

This allows us to define a new projection P(cr, k) by 

P(a,k) = M(a,k)( lN Q + Jjlf^r 1 

and next 

P(a, k, x) = T(cr, k, x, 0)P(a, k). 

The main interest of these definitions is that we have lZ(P(a,k)) = lZ(P + (a,k,0)) and 
1Z(I — P(cr,k)) = TZ(P~(a, k, 0)). Therefore thanks to (|4.11j) . we have for every x that 
lZ(P(a, k, x)) = TZ(P + (a, k, x)) and similarly that 

K(I- P(a,k,x)) =n(P-(a,k,x)). 
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Consequently, we have the estimates 

(4.13) \T(a, k, x, x')P(a, k, x')\ < Ce"^^^ , x, x' E R, x > x', Va E K, 

(4.14) \T(a, k, x, x'){I - P(a, k, x'))\ < Ce a{x ~ x,) , x, x' E R, x < x', Va E K. 

By using this property, the unique bounded solution of (|4.10p reads by Duhamel formula 



V(x) = I T(a, k, x, x')P(a, k, x')M(x') dx' - \ T{a, k, x, x')(I - P(a, k, x'))M{x') dx 

J —OO J X 

and hence, we get thanks to (|4.13|) . (|4.14p that 

\V{x)\ <C [ e-^ x - x '\\E.(x')\dx' 



which yields by standard convolution estimates 

\V\ L 2 <C\M\ L 2, 

We next estimate higher order derivatives. Write 

d s x +1 V = Ad s x V + [d s x ,A]V + d s x M . 

By considering [d x , A] V as part of the source term and by using the Duhamel formula, we 
get 

\V\ H * <C\M\ H s. 

This yields 

K| s < C|F|, fc+1 . 

This ends the proof of Lemma 14.21 

4.7. Proof of Lemma 14.31 We study again the equation 

aw = J(ik)Lw + J(ik)S(ik)w + J(ik)F. 
Again, we can apply R(cr, k) to get 

(4.15) aR{a,k)w - R(a, k)(j(ik)Lw + J{ik)S{ik)w) = R{a, k)J(ik)F. 

To solve (|4.15|) . we use a method close to the one used in [2Uj in a different context. The 
problem is that in estimates (|4.13|) . (|4.14|) . we have that a » p(k,K) may degenerate for 
k ~ 0. The convolution estimate 

\\e- a w*m\\v< nll/lb 

\a\ 

gives the rate of degeneration. The strategy is to write the solution w as a sum of two 
pieces. The first piece satisfies the needed estimate thanks to the Id assumption (hence no 
degeneration in the limit k — ► 0), while the second piece satisfies an equation of type (|4.15|) 
with a source term vanishing as |Re n(k, a)\ in the limit k — ► 0. This exactly compensates 
the singularity in the convolution estimate. 
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To be more precise, we seek the solution of (I4.15D under the form 

(4.16) w = u + v 
where u solves 

(4.17) aR(a, 0+)u - R(a, 0+) J{0)Lu = R{a, 0+) J(0)F 
and hence v solves 

(4.18) aR{a,k)v - R{a, k)(j{ik)Lv + J(ik)S(ik)v) = 
'r(ct, k)J(a, k)S(ik)u + a(R(a, k) - R(a, 0+))it 

+ (R{a,k)J(ik) - R(a,0 + )J(0))Luj + (R(a,k)J(ik) - R(a, 0+) J(0))F := H. 

The main interest of this manipulation is that the source term of (14.180 now vanishes 
thanks to (14. 8p when A; — ^ if A OQ (a, k) has an eigenvalue of vanishing real part. 
To solve (|4,17p . we can choose u as the solution of 

au - J(0)Lu = J(0)F. 

Since we assume that D does not vanish on /C, we can use Lemma 14.21 to get 

(4.19) \u\ s < C\F\ s+q . 

Thanks to the assumption (|4.8j) . this implies that the source term in (I4.18h satisfies the 
estimate 

(4.20) \H(a,k)\ s < Cp(k,K) \F\ s+q+qi 

for some q± > 0. To study (I4,18h . we can use the block structure (14. lh to get 

v 2 = E(a,k)- 1 (P 2 (a,k)v 1 + H 2 ), P^a, k) Vl = H x . 

Since by assumption the operators E and P 2 have a continuous extension to /C x [0,M], 
we get 

\v 2 \ s < c[\v\\ s+rn -i + \F\ s+ i +q+qi ^j 
uniformly for (<r, k) £ K, x (0, M]. Consequently, we only need to study the equation 

P 1 (a,k)v 1 = H 1 

to get the result. As in the proof of Lemma [321 we rewrite this equation as a first order 
system 

(4.21) V x = A{a,k,x)V + U. 
To get the existence of exponential dichotomies for 

(4.22) V x = A(a,k,x)V 
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on R + and K_ when k 7^ with a good control of C and a, we can use the conjugation 
Lemma of [23]. Thanks to Lemma 2.6 of [23] . there exist conjugators W±(x,a,k) such 
that W±(x,a,k) are invertible for every (a, k) with (a, k) with Re cr > 0, k G [0, M] and 
x G R± with a uniform bound of W± and W^ 1 and the property 

W± = ld + 0{e~ ±ax ) 

when x tends to ±00. Moreover, for every V solution of (|4^2]) . V 1 = W±~ 1 V solves 

(4.23) (Vi) a! = A 00 ((7,fc)Fi. 

Since for k 7^ , the spectrum of y4oo(c, A;) does not intersect the imaginary axis, the 
autonomous system (|4.23p has an exponential dichotomy on M, for k 7^ 0. Namely, there 
exists Poo (cr, k) and C > such that 

(4.24) \e xAoo ^ k) PooU\ < Ce- a ( k)x \U\,Vx>0,VU£C N 

(4.25) \e xA °°( a > k \l - P^U] < Ce a{k)x \U\, Vx < 0, VUeC N . 

where we can take a(fc) = /9(fc, K)/2. Moreover, P^ can be continued up to k = 0. Thanks 
to the conjugation property, we have 

(4.26) T(a,k,x) = W±(a, k, x)e xA °°( a ' k) W±{a, k, 0) _1 , i£l± 

and hence the projections P±(a,k,0) which define the exponential dichotomy for (14.221) 
are given by 

P + (a,k,0) = W+(a,k,0)P oo W+(a,k,0)~ 1 , P_(a,k,0) = W_(<r, k, 0)(Id-P oo )W-(a, k, 0) -1 . 

Since by assumption, the Evans function D does not vanish up to k = 0, we still have that 

KP+ (a, k, 0) TIP- (cr, k,0)=C N . 

Thanks to (|4~24jh (|4~25j) and (g]2gD , we thus get that (|4*T3]) . (gUD are still true for a £ IC 
and |/c| < M, fc 7^ with C independent of fc and a = a(k). 

By using again Duhamel formula and convolution estimates, we get for the solution of 

(HSU) 

in < 4m- 

and hence, we can use (|4,20p to get 

Hs 1,8+9+91 " c|F|s+9+91 

for k 7^ 0. This ends the proof. 



transverse instability 19 

5. Criterion for the existence of multipliers 

In this section we prove a criterion for the assumption of Section 12.3.41 

Lemma 5.1. Suppose that for every s>2 there exists a symmetric operator K s , bounded 
on I? such that 

E s = ~d x [J(ik),R]d x - S -(d x J{ik) [d x ,R] + [d x ,R]*J(ik)d x ) + ±[K„, J(ik)L ] 

is an operator of order 1, i.e. there exists C s (k) > with \E s u\ < C s (k)\u\i. Then, we 
have that there exists M s such that (2.11) and \2. 18\) hold. 



This general criterion can be used in a very simple way when J{ik) is a zero order 
operator, i.e. J{ik) G B(L 2 ). Indeed, we notice that in such a situation the second term 
in E s is already a first order operator. We will prove the following corollary: 

Corollary 5.2. Assume that J(ik) G B(L 2 ) and that L = -d 2 + L with L G B{H l ,L 2 ). 
Then K s = R verifies the assumption of Lemma \5.1\ i.e. E s is a first order operator and 
hence there exists M s such that (|2,17p . (|2.18p hold. 

5.1. Proof of Corollary 15.21 We check that the assumption of Lemma 15.11 is verified 
with K s = R. As already noticed the second term in the definition of E s in Lemma 15.11 is 
already a first order operator since J is a zero order operator. Next, by the assumption 
Lq = — d 2 + L, we notice that 

[K s , J(ik)L ] = -d x [K s , J{ik)]d x + E 

with E a first order operator. This proves that E s is indeed a first order operator with 
the choice K s = R. 

5.2. Proof of Lemma 15.11 For s > 2, we define the symmetric operator 

m s u = (-lyd^u + i-iy^d^-^Ksd^u). 

Thanks to the L 2 boundedness of K s the assumption (j2. 17[) is clearly satisfied. Let us 
next check (I2.18p . For that purpose, we need to evaluate the quantity 

Re({J{ik)(L + S{ik))u, M s u). 

Since J(ik)Lo is skew-symmetric, we have 

(5.1) Re( J{ik)L u, (-l) s d 2 x s u) = Re(J{ik)L d s x u, d s x u) = 0. 

We can also write 

Re(j{ik)Ru,(-l) s dl s u) = Re(J{ik)Rd s x u,d s x u) 

+ sRe( J(ik)[d x , R]d^\ <9» + {Cu, d s x u) 
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where \Cu\ < C\u\ s -\. Furthermore, since J(ik) is skew symmetric and R symmetric, 
R&(j{ik)Rd s x u,d s x u) = -Re((J(ik)R- RJ{ik))d s x u,d s x u) 

= - hle(d x [J(ik),R] d x d x -\ d^u) . 

Therefore, we finally find 

(5.2) Re(j(ik)Ru,(-l) s dl s u) 

= ~ Re{d x [j(ik), r] d x %r\ d b x - l u) 

~({d x j{ik) [d x ,R] + [d x , r]* j{ik) d x )d s x r l u, d s x r l v) + o{i)\u\ s |«| s _i- 

Next, since J(ik)S(ik) is skew-symmetric, 

(5.3) Re(J(ik)S(ik)u, (-l) s dl s u) = (J(ik)S(ik)d s x u, d s x u) = 0. 
Since J(ik)Lo is skew-symmetric and K s symmetric, we also have 

Re(J(iA;)Lou,(-l) s ~ 1 ^- 1 ^^- 1 u) = Re(K s J(ik)L d^u, d x ~ l u) 

(5.4) = i([K a ,J(ik)L ]d'- 1 u,a^ 1 u). 

Since K s is bounded on L 2 and J(ik) of order one, we have that 

(5.5) Re(J(ik)Ru, (-l) 5 " 1 ^" 1 K s d s ~ l u) = 0(l)\u\ s \u\ s -i. 

Next, since by the assumptions of section [2.2.21 we have in particular that J(ik)S(ik) d x £ 
B(H 2 , L 2 ) and since K s is bounded on L 2 , we have 

(5.6) Re(J(iA;) 1 S(iA;)n,(-l) s ~ 1 a^- 1 K s a^- 1 u) = 

Re(J(ik)S(ik)d x d s ~ 2 u,K s d s x r l u) = 0{l)\u\ s \v\ a -i. 

Collecting fl5J]), <^), ([O]), (JED), ([53]), (HUD, we infer that 

Re({J{ik)(L + S{ik))u,M s u) = {E s d s - l u,d s x r l u). 

In view of the assumption on E s , we obtain that the assertion of the proof of the lemma 
holds. 

6. Proof of Theorem Q] (periodic perturbations) 

The general strategy of the proof is inspired from the work of Grenier [12] in fluid 
mechanics. 
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6.1. Construction of a most unstable eigenmode. By the assumption there exists 
an unstable mode with associated transverse frequency ko ^ 0. The first step of the proof 
is to find the most unstable eigenmode. This means that we look for an unstable mode 
with associated transverse frequency mko, m £ Z such that the associated amplification 
parameter a has maximal real part. This is indeed possible thanks to the following lemma. 

Lemma 6.1. Consider the problem 



There exists K > such that for \mko\ > K there is no nontrivial solution of 16.1]) with 
Re(a) / 0. 

In addition, for every k ^ there is at most one unstable mode with corresponding trans- 
verse frequency k. 

Proof. Recall that by assumption if U solves dfiTj then U belongs to ^(K;^) n V s . 
By taking the real part of the scalar product of (|6.1j) with LU + S(imko)U, we get the 
following "conservation law" 

(6.2) = Re(a((U,LU) + (U, S(imk )U))) = Re{a)({U, LU) + (U, S{imk )U)). 

Indeed, since J(imko) is skew-symmetric, we have Re (J(imko)u, u) = for every u £ H°° 
and we have also used that L and S(ik) are symmetric. Thanks to (12.81) . we get that for 
\mko\ > K there is no nontrivial solution of (|6.2j) with Re(cr) ^ 0. 

Let us now prove the second assertion of the lemma, i.e. we shall prove that for k ^ 
there is at most one unstable eigenmode with corresponding transverse frequency k. 
Thanks to (16. lj) . we first notice that an unstable eigenmode must be in the image of 
J(imko), consequently, since J(ik) is into, we can write U = J(ik)V with V S H°°(W; C N ) 
a nontrivial solution of 



Note that is a symmetric operator. Next, we observe that the operator J{ik)LJ{ik) 
has at most one positive eigenvalue. Indeed, by contradiction, if J{ik)LJ{ik) had an 
invariant subspace E of dimension at least 2 on which the quadratic form J{ik)LJ{ik) is 
positive definite, then the quadratic form (Lu, u) would be negative definite on J(ik)E 
and since J(ik) is into J{ik)E is also two-dimensional. This gives a contradiction since L 
has only one simple negative eigenvalue. 

Next, we can also prove that has at most one simple positive eigenvalue. Again, if 
Mfc has an invariant subspace E of dimension at least 2 on which the quadratic form 
(MkU, u) is positive definite then there exists u G E n 7^ {0} where tp is the only 
positive eigenvalue of J(ik)LJ(ik). Since on (ip)' L , J(ik)LJ(ik) is non positive, and S(ik) 
is positive, we get 



(6.1) 



all = J{imk ){LU + S(imk )U), U £ L 2 (M; C d ) . 



(6.3) 



aJ(ik)V = (j{ik)LJ{ik) + J{ik)S{ik)J{ik)) V = M k V, k = mk . 



(M k u,u) = (J(ik)LJ(ik)u,u) + (J(ik)S(ik)J(ik)u,u) < 
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which yields a contradiction. Consequently has at most one positive eigenvalue. Fi- 
nally, we can use \26\ Theorem 3.1] to get that J(ifc) _1 M/ c has at most one unstable 
eigenvalue. Consequently, for k ^ 0, there is at most one unstable a for which (16. lj) has a 
nontrivial solution. This completes the proof of Lemma 16.11 □ 

By the assumption (12. 12ft . we know that for k = 0, there is no unstable eigenmode. We 
consider the finite set A of integers m such that ko < | mko\ < K, where K is provided by 
Lemma 16.11 Again by Lemma l6.1( for every m S A there is at most one unstable mode 
with corresponding transverse frequency mfco- Moreover, by the assumption of Theorem Q] 
for m = 1 there is an unstable mode. We now take the unstable mode U corresponding 
to mo ^ A with maximal real part of the corresponding amplification parameter which we 
note by <jq. We set 

u°(t, x, y) = e aot e imokoy U + e * f °V im ° fco2/ U = 2Re(e aot e im ° koy U) . 

To prove Theorem [H we shall use Q + 5u° (0) as an initial data for (|2.4j) . Thanks to 
our assumptions of section 12.41 about the nonlinear problem, the problem (12. 4j) is locally 
well-posed with data Q + Su°(0). 

6.2. Construction of an high order unstable approximate solution. Denote by 
Fj G C°°(lR d ;lR), 1 < j < d the derivative of F with respect to the j'th variable, i.e. 
VF = (Fi, ■ ■ ■ ,F d ). For a G N d , we set 

(6.4) F a =(ff*Fi(Q),.-. ,d a F d (Q)). 

Let us look for a solution of (|2.4p under the form u = Q + Sv, where 5 e]0, 1]. Recall the 
Taylor formula 

f(x + y)-f(y)= E ^/(y) + (iV+l) EI [\l-t) N d a f(tx + (l-t)y)dt, 

l<|a|<AT |a|=JV+l u 

where > 1 and / G C°°(lR d ;lR). In what follows, we shall also use that for s > 2, M s is 
an algebra , and that ||/(ii)|| s < ^-(ll^lls)) where A : M — > R + is a continuous function. We 
obtain thus that for every M > 1, v solves the equation 

(6.5) 5d t v = J{d y )(6(L + S{d y ))v+ S^v a F a + d M+2 R MtS (v)^, 

2<\a\<M+l 

where F a is defined by (|6.4p and Rm,8 satisfies for s > 2 

V<5 e]0, 1], Vtj G EI S , ||i?M,<5WI| s < ||u||f +2 A M (||5i;|| s ) , 
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where Am '■ K — > K + is a continuous function. We define as the space 

K 

V£ = {« : u= ^ u o e ijmokoy , Uj G # S (M)} 

j=-A 

and we define a norm on V£ by = sup^ \uj\ s . Let us notice that u° is such that 

u° G for all s G N. Following the strategy of [T2|, for s > 1, we look for an high order 
solution under the form 

M+l 

(6.6) u ap = 5u°+Y, S k u k , u k G V k s +1 

k=2 

such that u k t=Q = and M > 1 is to be fixed later. 

By plugging the expansion in (|6.5p and by cancelling the terms involving 5 k+1 , 1 < k < 
M, we choose u k so that n fc solves the problem 

(6.7) d t u k = J(d y )(Lu k +S(d y )u k )+J(d y ) ( E «f •■■«?) ^« 

2<|o|<fc+l |/3|=fe+l-|a| 

where Uj j stands for the j'th coordinate of and with the initial condition u k t=Q = 0. 

Note that the term involving 5 cancels thanks to the choice of u° (while the term in front 
of 5° is absent in (|6.5p thanks to the choice of Q). Thanks to our assumptions u k is a 
solutions of a linear equation which is globally defined. Indeed, we can define exp(JLo) 
via the Fourier transform and then treat the problem for u k perturbatively. Moreover 
u k G 1 for every set, The main point in the analysis of u ap is the following estimate. 

Proposition 6.2. Let us fix an integer M > 1. Letu k be the solution of (j6.7j) . < k < M . 

Then for every integer s > 1 there exists a constant Cm,s such that we have the bound 

(6.8) \u k {t)\ V s +i <C M ,se {k+1)Re{ao) \ Vt>0. 
As a consequence there exists Gsff for all s such that 

d t (Q + u a P) - J{d y ) (L (Q + u a P) + VF(Q + u ap ) + S(d y )(Q + u ap )) = J(8 y )G 
and for < t < log(l j 8) / Re(oo) and s > one has the bound 

\\J(d y )G(t)\\ s < C M ,s5 M+2 e (M+2)i?e(<Jo)i , 
where Cm,s is independent oft G [0, log(l/<5)/i?e ctq] and 5 G]0, 1]. 

By an easy induction argument Proposition 16.21 is a consequence of the following state- 
ment. 
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Proposition 6.3. There exists q £ N such that for s > 3, if f(t) £ V^ q satisfies 
(6.9) |/(t)| V '+« ^ C^ s e 7i , 7 > 2Re(a ) 



K 



then the solution u of the linear problem 

(6.10) 8 t u = J(d y )(Lu + S(d y )u) + J(d y )f, u /t=0 = 0. 
belongs to and satisfies the estimate 

(6.11) K*)|V£ < C^e^, Vt > 0. 

Since / has a finite number of Fourier modes in y, Proposition 16 . 31 is a direct consequence 
of the following Id result. 

Proposition 6.4. There exists q £ N such that for j such that j/(moko) £ Z with 
|j|/( m o^o) < ^ (where K is provided by Lemma \6. 1\) and s > 3, if we suppose also 
that 

(6.12) \fj(t)\s+q < Cj^\ 7 > 2Re(a ) 
then the solution of 

(6.13) d t v = J(ij)(L + S(ij))v + J(ij)fj, v /t=0 = 
satisfies 

\v(t)\. < C jt .e* . 

We shall prove below that Proposition I6.4l is a consequence of the following key resolvent 
estimate. 

Proposition 6.5 (Resolvent Estimates). Let'jo be such that Re(o~o) < 70 < 7. Suppose 
that w solves the resolvent equation 

(6.14) (70 + ir)w = J(ij){L + S(ij))w + J(ij)H 

with \j\/{moko) < K. Then there exists q £ N such that for s > 1 an integer there exists 
C(s,^q, K) > such that for every r, we have the estimate 

(6.15) \w(t)\ s < C(s, 7o ,K)\H(r)\ s+q . 

6.2.1. Proposition \6.5\ implies Proposition \6.4\ For T > 0, we introduce G such that 

G = 0, t < 0, G = 0, t > T, G = fj, t £ [0, T] 
and we notice that the solution of 

$5 = J(ij)(L + S(ij))v + J(ij)G, v /t=0 = 
coincides with v on [0, T]. Indeed, w = v — v solves for t £ [0, T] the equation 

d t w = J(ij)(L + S(ij))w, w /t=0 = 0. 
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By taking the real part of the scalar product of this equation with w, we get by skew- 
symmetry of J(ij)Lo, J(ij)S(ij) and J(ij) that : 



d . 

— \w 



2 



2 Re (J(ij)Rw,w) = -(w, [R, J(ij)]w). 



dt 

Consequently, thanks to (|2.6jh we get 

^\w\ 2 <C\w\ 2 
dt 1 1 ~ 11 

and hence, after integration in time , we find that w = on [0, T] . It is therefore sufficient 
to study v. Next, we set 

w(t,x) = Cv (70 -Mr), H(t, x) = CG(j + ir), (r, x) 6 M 2 



where £ stands for the Laplace transform in time : 



£/(7o + ir) = / 

JO 



oo 

—fot—irt 



fit) dt. 







By using Proposition 16.51 and Bessel-Parseval identity, we get that for every T > 0, 

e- 2l0t \vit)\ 2 s dt< / e- 2 ^ ot \vit)\ 2 dt = C / h(r)| 2 dr 
Jo Jr 

<C I \Hir)\ 2 s+q dr= [ T e- 2 ^\ fj (t)\ 2 +q dt 
Jr Jo 

and finally thanks to (|6.9p . we get 

(6.16) j T e- 2 ^ ot \vit)\ 2 dt<C [ T e 2 ^ o)t dt < Ce 2 ^~ lo)T 



since 70 was fixed such that 7 > 70- To finish the proof, we shall use a crude H s estimate 
for the equation ()6. 13|) . By using that J(ij)Lo an d J(ij)S(ij) are skew-symmetric together 
with (|6.12[) . we obtain 

j t \vit)\ 2 < c(|/ J (t)| 2 +1 + 2Re {J(ijKiRv),d% 



a, v 

\a\< 

< C\vit)\ 2 +1 + Ce 2 ^, 



s+1 

where we have used that J(«j) is an operator of order 1. It is possible to have a better 
estimate involving only \v\ 2 in the right-hand side, but it is useless here. Next, for < 
70 < 7, we get 

|(V 270t N*)lf) < C(e^\vit)\ 2 +1+ e 2 ^). 
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Therefore, we can integrate in time and use (|6.16l) (with s + 1 instead of s) and the fact 
that 7 > 70, to find 

e~ 2yot Ht)\ 2 s < Ce 2 ^~^K 
Therefore, we have shown that Proposition 16.51 implies Proposition 16.41 

6.3. Proof of Proposition 16.51 We shall deal differently with the large and bounded 
temporal frequencies. Indeed, Proposition 16.51 is a consequence of the following two state- 
ments. 

Lemma 6.6. For every 70 > and K £ N, there exists M > such that for every s > 1, 
there exists C(s, *yo, K) such that for |r| > M, s > 1, we have the estimate 

(6.17) \w(t)\ 2 <C(s, 1o ,K)\H(t)\ 2 +1 . 

Lemma 6.7. There exists q £ N such that for every 70, Re o~q < 70 < 7, s > 1, K G N 
and M > 0, i/iere exists C(s,jq,K,M) such that for |r| < M and s > 1, we have the 
estimate 

(6.18) \w{t)\ 2 s < C(s, l0 ,K,M)\H(r)\ 2 s+q . 

6.3.1. Proof of Lemma \6.(A We first prove (|6,17p for s = 1. Recall that the equation (|6.14p 
reads as follows 

(6.19) (70 + ir)w = J(ij)(L + S(ij))w + J(ij)H. 
By the assumption (|2.3p . we can define an orthogonal decomposition: 

(6.20) w = otip-i + wq + wj_ 
where 

(6.21) Lip-i = fj,(p-i, /-i < 0, Lwq = 0, (Lw±,w±) > co\w±\f, cq > 0. 

Indeed, to obtain the last estimate, we have used that by the assumption 12.31 we have the 
lower bound 

(6.22) (Lw ± ,w ± ) > c \w ± \ 2 , 

but thanks to the decomposition L = Lq + R, and the lower bound (|2.2p . we also have 

(6.23) (Lw±,w±) > C^w^l - C\w±\ 2 

for some C > since R is bounded on L 2 . Consequently, we can consider A (j6.22j) + (16.231) 
with A such that Acq > C to get the claimed in (|6.2ip lower bound. We normalize (p-± 
such that = 1 

Note that by the assumption after (|2.3p . (f-i and wq are smooth. Indeed, wq is smooth 
since the kernel of L is spanned by a finite number of smooth eigenvectors and by expaning 
wq on a smooth basis, we also have that for every s > 1, there exists C s such that 

(6.24) |w |s < C s |u;o|2 < C s \w\i. 
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Again, we use the conservation law 

(6.25) 70 {(w,Lw) + (w,S(ij)w)) =Re((J(ij)H,(L + S(ij))w)). 
Consequently, we can use (|6.20[) . (|6.2ip to get 

7o(/iQ ! 2 |^_i|f + co\w ± \l + \w\ 2 s(ij) ) < \(J(ij)H,S(ij)w)\+\(J(ij)H,Lw)\. 
To estimate the right hand side, we first use (|2.7p and the skew-symmetry of J to get 

\(J(ij)H,S(ij)w)\ = \(H,J(ij)S(ij)w)\ < C(j)\H\ \w\ s(ij) . 
Next, we notice that thanks to (|2.2p and Cauchy-Schwarz, we have 

(6.26) \(u, Lqv)\ < C\u\\ Vu, v. 
This yields thanks to (|2.5p : 

\(J(ij)H,Lw)\ < |J(y)fl"|i |to|i <C\H\ 2 \w\l 
We have thus proven that 

(6.27) j (fia 2 \ip-i\ 2 + c \w±\ 2 + \w\s(ij)) ^ Cl^b^Mi + C(j)\w\ s ^\ 
By using the inequality 

(6.28) ab<ea 2 + -^b 2 , Ve > 0, V(a,6)eIR 2 , 

with e small enough, we can incorporate l^ls^-) in the left hand side of (|6.27p and arrive 
at 

(6.29) \w ± \l + \w\% m < C(\a\ 2 + \H\ 2 + \H\ 2 Hi)- 

In what follows C is a large number which may change from line to line and depends on 
7o and K but not on r. The next step is to estimate a and wq. We use the decomposition 
(|6.20p and take the scalar product of (I6.19|) with aip-\ and with wq respectively to get 

(7o + ir)\a\ 2 = -a({w,LJ(ij)(y-{)) + {J(ij)S(ij)w,ip-i) + {J{ij)H, <^_i)) 

(70 + ir)\w \ 2 = -(w,LJ(ij)w ) + (J(ij)S(ij)w,w ) + (J(ij)H,w ) 

and hence, we can take the modulus and add the two identities to get thanks to (16.241) 
and (ETYJ) that 

(70 + M)(M 2 + Kl 2 ) < c(H 2 + Kl 2 + Kl 2 + \w\ 2 m + \h\ 2 ) 

which we can rewrite as 

(6.30) (70 + \t\ - C)(\a\ 2 + Kl 2 ) < C(|^_l| 2 + \w\ 2 s{ij) + \H\ 2 ). 
Combining (|6.29p and (|6.30p . we infer that there exists M > such that for |r| > M, 

(6.31) \w\\ + \w\ 2 s{ij) < C(\H\ 2 Iw]! + \H\ 2 ). 
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To conclude, we use again the inequality (|6.28[) and we obtain 
(6.32) \w\\ + \w\ 2 sm < C\H\ 2 2 . 

This proves (|6.17p for s = 1. Note that moreover (|6.32p gives a control of IHs(y) which is 
interesting when j ^ 0. 

In order to estimate higher order derivatives, we use the operator M s defined in section 
12.3.41 By taking the scalar of product of (|6.19j) by M s u and taking the real part, since 
M s is self- adjoint, we find 

yo(w, M s w) < Re(( J{L + S{ij))w, M s w) + (JH, M s w)) 

and hence we find thanks to (|2.17p . (|2.18p . 

7o(Hs - C \ w \s-i) < C \ W U \w\s-i + C\JH\ s \w\ s . 
This yields by a new use of the Young inequality (|6.28[) 

Hs < C{Hs-i + \JH\l) 
and hence, thanks to the assumption on J, we have 

\w\ 2 s <C(\w\ 2 s _ 1 + \H\ 2 s+1 ). 
An induction argument completes the proof of Lemma 16.61 



6.3.2. Proof of Lemma 6.7 The assertion of this lemma is a part of our assumptions. 
Indeed, for a = 70 + it , |t| < M and every j, \j\ < k, we have by choice of 70 that there 
is no unstable modes on this line which is equivalent to D(cr,j) / 0. Consequently, the 
assumption (I2.14p gives the result. 

The proof of Proposition 16.21 is therefore also completed. 



6.4. Nonlinear instability (end of the proof of Theorem [T]) . We look for a solution 
of (|2.4p in the form u = Q + u ap + w. Then the problem for w to be solved is 



d t w = J(d y ){L w + VF(Q + u ap + w)- VF(Q + u ap ) + S(8 y )w) - J(d y )G 
with zero initial data, where thanks to Proposition 16.21 
(6.33) \\J(d y )G(t, -)\\ s < C M ,^ A/ +V M+2 ) Re ^, 

as far as < t < T s , where 

T 5 _ fog( K /^) 
Re(ao) 



TRANSVERSE INSTABILITY 29 

with k G]0,1[ small enough, the smallness restriction on k to be fixed in this section. 
Thanks to our nonlinear assumption 12.41 and the structure of u ap , w is defined for small 
times. Next, since J{d y )LQ and J(d y )S are skew symmetric, w enjoys the energy estimate 



d_ 



wit)fs< E / {d a (jDVF(Q+u a P(t)+aw(t)y W (t)),d a w(t)^da+\\JG(t)l 

\a\<s 



W 



where ((•,•)) denotes the L 2 (K x T a ) scalar product. Let us define a maximum time T* 
such that 

T* =sup{T : T <T 5 , and Vt G [0,T], |K*)|| a < 1}. 
(T* is well-defined since w(0) = 0). Consequently, we can use (|2,20p . with s > sq to get 

(6.34) \\w(t)\\ 2 s < / (||JG(r)|| a ||«;(r)|| a +a;(C + /6C7 Jlf>8 )Kr)||2)dr, 

J o 

provided also that £ < T*. Combining (|6.33|) and (|6.34|) . we obtain that for t £ [0,T*[, 
\\w(t)f s < co(C + kC m ,s) f T \\w(T)\\ 2 s dr + c M ^ M +^e 2( ~ M+2 ^)t _ 



We take an integer M large enough so that 2(M + 2)Re(o"o) — w(C) > 2. At this place we 
fix the value of M. We then choose k small enough so that 

2(M + 2)Re(cj ) - 1 > w(C + kC m , s ) - w(C) . 

Such a choice of k is indeed possible thanks to the continuity assumption on to. By a 
bootstrap argument and the Gronwall lemma, we infer that w(t) is defined for t £ [0, T 5 ] 
and that 

sup \\w{t,-)\\ s <C M ^ M+2 . 

0<t<T 6 

In particular 

(6-35) \\w(T\-)\\ l2{RxJl) <C M)S k m+2 . 

Let us denote by II the projection on the nonzero modes in y. For an arbitrary w £ T 
(an L 2 (R) function depending only on x) one has Tl(w) = 0. On the other hand the first 
term of u ap satisfies II (it ) = n° and therefore using (|6.8p 

M 

\\U{u ap (t,-))\\ L 2 > c s 5e R < a ^ -E<5 fc+1 ||n(n fc )||L2 

k=l 

M 

k=l 
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Therefore for k small enough one has 

(6-36) ||n(^(T 5 ,.))llL 2 (RxT L )>^- 

Using (|6.35|) and (|6.36|) . we may write that for every w £ 

\\u s (T 5 ,-)-w\\ L2 > \\U(u 5 (T 5 ,-)-w)\\ L 2 

= \\U(u s (T s ,.)-Q(.))\\ L2 

= \\n(u a P(T 5 ,-) + w(T 5 ,-))\\ L2 

> C -f-\\U(w(T s ,.))\\ L2 

> c -f-\HT 5 ,-)\\^ 

A final restriction on k may insure that the right hand-side of the last inequality is bounded 
from below by a fixed positive constant rj depending only on s (in particular ry is indepen- 
dent of 5) . This completes the proof of Theorem [TJ □ 



7. Proof of Theorem [2] (localized perturbations) 
The first step is to find the most unstable eigenmode which solves 
(7.1) aU = J(ik)(LU + S(ik)U). 

Since now k is in R it is slightly more complicated. We begin with a few preliminary re- 
marks which allow to reduce the search for unstable eigenmodes in a compact set. Thanks 
to Lemma 16.11 and f|2.8[i , we already now that unstable eigenmodes must be seek only for 
k £ [0, K], where K is fixed by assumption (|2.8j) . Moreover, by taking the scalar product 
of (|7.ip by U, and then taking the real part, we get 

Re{a)\U\ 2 < Re (J(ik)RU,U) 

since J(ik)L$ and J(ik)S(ik) are skew symmetric. Since J(ik) is also skew symmetric, we 
also have 

Re (J(ik)RU,U) = ([J(ik), R]U,U) 
and hence, thanks to (|2.6p . we obtain 

Re(a)\U\ 2 < C\U\ 2 . 

Consequently, there is no nontrivial solution for Re(cr) sufficiently large. Next, by using 
the result of Lemma l6.6l for H = 0, we find that for every 70 > 0, there exists C(7o, K) such 
that there is no nontrivial unstable solution of (|7.1|) for Re<r > 70 and |Im<r| > C{^q,K). 
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Now, let us assume that the unstable eigenmode given by our assumption is such that 
Re a = 5. Then thanks to the previous remarks, the most unstable eigenmode has to be 
seek in the compact set 

(a,k) elZ = {5/2 < Rea < C, \Ima\ < C{5,K)\, \k\ < K). 

Moreover, we have an unstable eigenmode if and only if (<r, k) is a zero of the corresponding 
extended Evans function D(a,k) which is an analytic function in {Recr > 0} x I, We 
have already proven that for each k there is at most one zero with Recr > 0. By Rouche 
Theorem, if there exists (o"o,fco) with Recro > such that D(ao,ko) = 0, there exists 
a vicinity of ctq and ko such that for each k, there is exactly one zero a = u{k) of D. 
Moreover, k — > cr(k) is analytic. Indeed, we have the explicit expression 

a(k) = C [z^^dz, 
K ' Jr D(z,k) 

where T is a circle which contains a® and C is a constant and hence the analycity of D 
gives the analycity of a. If we define ft = { k, 3 <r, Re a > 5/2, D(cr, k) = 0} , this proves in 
particular that f2 is an open bounded set (and non empty thanks to the assumption of the 
existence of an unstable mode) of K. One can decompose f2 as Q = \J m I m where I m are 
disjoint, open and bounded intervals which are the connected components of $7. On each 
I m the above considerations prove that there exists an analytic function k — > c(k) such 
that cr{k) is the only zero of D in Recr > 0. We shall prove next that k — > Reo"(/c) has a 
continuous extension to I m . Indeed, if k n is a sequence converging to an extremity k of 
I m , since a(k n ) is bounded (a(k n ) S Tl), then we can extract a sub-sequence not relabelled 
such that a{k n ) tends to some a. Moreover, we also have Reu > 5/2, and D(cr, k) = 0, so 
a is the only unstable zero of D(-,k). This allows to get that limfc^ Ki fc e / m a{k) = a and 
hence to define a continuous function on I m . Finally, we also notice that if dI m PidI m i ^ 0, 
then the continuations must coincide again thanks to the fact that there is at most one 
unstable eigenmode. Consequently, we have actually a well-defined continuous function 
k — > cr(k) on £1 which is a compact set. This allows to define the most unstable eigenmode 
as 

do = Re<r(/co) = sup{<ro(/e) = Re<r(/c), k £ ft}. 

Note that, since we have assumed that there exists an unstable mode, do is positive and 
also that ko ^ thanks to the assumption (|2.15p . Moreover, ao(k) is an analytic function 
in the vicinity of ko and hence, there exists m > 2 

(7.2) [Re(a)]'(k ) = ■■■ = [Re^)]^- 1 )^) = 0, [Re(a)]^(k ) + 0. 

Let I be an interval containing ko which does meet zero. For k G /, let us denote by U(k) 
the unstable mode corresponding to transverse frequency k and amplification parameter 
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a(k). Then we set 

x, y, k, a(k)) = 2Re(e a ^ t e iky U(k)) , 
where the dependence of x of $ is in U(k). Further we define 

u°(t,x,y) = J &(t,x,y,k,a(k))dk . 
The function u° is the first term of our approximate solution, i.e. it is a solution of 

(d t - j(dy)L ~ j{dy)S{dy))U = . 

Recall that <7o = Ke(a)(ko). Thanks (|7.2p . we can apply the Laplace method (see e.g. 
[HJ [9] ) and obtain that for every s > there exists c s > 1 such that for every t > 

(7.3) - 1 e°°' < ||u (V)l|ff«<*») < ; V ■ 

C S (l+t)5^ (l + t)2^ 

As in the previous section, we look for an approximate solution of the form 

M 

(7.4) u ap = 5(u° + J2 <^ fe ) > uk G l2 ( r2 ) > 

k=l 

where 5 <C 1 and M 3> 1 and u k , 1 < k < M are solutions of (|6.7p with zero initial data. 
Observe that the Fourier transform of u k with respect to y is compactly supported. Thus 
using the Fourier transform in y, the Laplace transform in t, and (|7.3p . we can deduce as 
in the proof of Theorem Q] the bounds 

(7.5) h k (t,.)\\ s < Cs * e^°t 

(1 + 1)~ 

from the following resolvent estimate. 

Proposition 7.1. Consider w(t) the solution of 

(70 + ir)w = J(ik)(L + S(ik))w + J(ik)H, ao < 70 < 2gq. 

Then there exists q > swc/i that for every integer s > 1, and every K > 0, there exists 
C(s,7o,-?0 such that for every k G IR\{0}, \k\ < K, and r £ ]R, we have the estimate 

(7.6) \w(t)\ 2 s <C(s, 10 ,K)\H(t)\ 2 



s+q- 



Proof of Proposition \7.1\ As in the proof of Proposition [631 we can split the proof of (|7.6 



into large |r| estimates and bounded |r| estimates. The proof of the large |r| estimate 
was already proved in Lemma 16.61 As already noticed, there is no difference between the 
continuous and discrete cases in k for this estimate. To treat the small |r| case, we use 
the assumptions of Section 12.3.31 By choice of 70 and thanks to the assumption (|2.15p , we 
can use fl2T6j) for K, = {70 + ir, |r| < M}. □ 
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With (|7.3p and (|7.5j) at our disposal, we may complete the instability proof as in the 
previous section. We choose T s > such that 

P T 6 a K 



[1 + T S }2^ S ' 

where k > is small enough to be fixed. Again, we write the solution of (|2.4p in the form 
u = Q + u ap + w with w solution of 

d t w = J(d y )(L w + VF(Q + u ap + w)- VF(Q + u ap ) + S{d y )w) - J(d y )G 

with zero initial data. Thanks to (|7.5p . we have that 

(Af+2)Re(<r )t 

(7-7) ||.7G(t, -)|| s < C M ,s5 M+2 I7+2 - > < t < T . 

(1 + i)^T 

Then thanks to our assumptions, w enjoys the energy estimate 

(7.8) \\w(t)f s < f (\\JG{t)\\ s \\w{t)\\ s + U (C + KC M , s )\\ W (r)f s ) dr, 

Jo 

provided also that t < T s and t small. Let us define T* by 

T* =sup{T : T<T 5 , and V £ G [0, T], \\w(t)\\ s < 1}. 
(T* is well-defined since w{0) = 0). Thanks to ([721) and (J7HD, we obtain that for t G [0, T* [, 

/•T 2(M+2)Re( ( r )t 

\\w(t)f s <u;(C + KC MtS ) / |Kt)||> + C m , s * 2(M+2) — • 

Jo (l + t)~*r 

We fix an integer M large enough so that 2(M + 2)Re(ao) — w(C) > 2. We then choose k 
small enough so that 

2(M + 2)Re(cj ) - 1 > w(C + kC A /, s ) - w(C) . 

Using the inequality 

t e 2(M+2)a T-ui(C+KC MtS )T q e 2(M+2)a t-u}(C+KC M , s )t 

(J T < 

(1 + t)~2^~ (l + t)"5S- 

a bootstrap argument and the Gronwall lemma, we infer that w(t) is defined for t G [0, T 5 ] 
and that 

ihtV)|| s <c m ,^ m+2 . 

In particular 

(7-9) \\w(T 5 ,-)\\ l2{RxJl) <C m ,sK M+2 . 
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Let Iq be a neighborhood of zero which does not meet I. Let us fix ip E Cg°(R) vanishing 
on Iq and equal to one on /. Let us denote by II the map acting on S'(R 2 ), defined via 
the Fourier transform as 

n(5(£i,6) = ¥>(&)fi(£i,&). 

The Fourier multiplier LT is bounded on L 2 (M 2 ) and we notice that the first term of u ap 
satisfies II(u ) = u°. Therefore, we have that 

pRe(o-o)* M 

||n(u*(*,-))llL« > Cs5- -^-^25 k+1 \\n(u k )\\ L2 

(l+t)3S fri 

e Re(v )t M e (fc+l)Re(a )t 

> C S <5- — - - Vft, s { W - "5+T-- 

(l + t)35I ^ (l + t)*^ 

Therefore for k small enough one has 

(7-10) ||n(^(T 5 ,-))ll^(]RxTO > 

Finally, since for w £ JF C 5'(IR 2 ), where which is defined in statement of Theorem [2] 
is the set of functions (or tempered distributions) which depends only on x, we have that 
H(w) = 0, by using (17.91) . (I7.10|) . we can write that for every w £ J 7 , 

\\u s (T s ,-)-w\\ L2 > \\U(u s (T s ,-)-w)\\ L2 

= \\n(u s (T s ,.)-Q(.))\\ L2 

= \\n(u a P(T 5 ,-) + w(T 5 ,-))\\ L2 

> C -f-\\U(w(T\-))\\ L2 

> c -f-\HT\.)\\ L2 

A final restriction on k may insure that the right hand-side of the last inequality is bounded 
from below by a fixed positive constant r/. This completes the proof of Theorem [5J □ 



8. Examples 



In this section we give a number of examples when our general result of Theorem [T] and 
Theorem [2] applies with an unstable mode generated by Lemma 13.11 
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8.1. The generalized KP-I equation. The Id model is the gKdV equation 

(8.1) u t = d x {-dl -u p ), p = 2,3,4, u : R -> R. 

For simplicity, we consider only the case of power nonlinearities but more general nonlinear 
interactions may be considered too. Equation (|8.ip has a solution of the form u(t, x) = 
Q(x — t) with Q smooth with exponential decay. We even have an explicit formula for Q 
namely 

(8.2) Q(x) = (P±±) ^ (sech 2 (fci^ ^ 



The solution u(t, x) = Q{x — t) describes the displacement of the profile Q from left to the 
right with speed one. One also has the solution 



i 



(8.3) u c (t, x) = cp- 1 Q{y/c{x - ct)), c> 

which correspond to a solitary wave with a positive speed c. We restrict our considerations 
only to speed one solitary waves since the case of arbitrary speeds can be reduced to speed 
one by a change of scale because of (j8.3|) . 

By changing x into x — t, we observe that Q is stationary solution of 

(8.4) u t = d x {-dlu + u-u p ) 
which fits into the framework of section 12.11 with d = 1, 

u p+l 

J = d x , L = -3l + Id, F(u) = — . 

p + l 

Obviously, the first assumptions of section [24] are matched. Moreover, we have, 

L = -d 2 x + Id - pQP-Hd, R = -pQP'Hd . 

The spectral condition (|2.3p on L is satisfied by Sturm-Liouville theory since Q' has only 
one zero (see [21 [3]). 

The transversally perturbed model is the gKPI equation which reads in the moving 
frame 

(8.5) u t = d x {-dlu + u-u p + d- 2 d 2 y u). 

Consequently, we have J(ik) = d x , S(d y ) = d~ 2 dy and hence S(ik) = —k 2 d~ 2 . 
The assumptions of section [2.2.11 are obviously met. In particular, since 

[R, J]w= P (Q p ~ 1 ) x w, 



(1X6|) is true. 

Next, one can also easily check the assumption of section 12.2.21 Note that 
k 2 \d~ 1 w\ 2 , hence, assumption (|2.7|) is satisfied with C(k) = \k\. 



:-!(> 



FREDERIC ROUSSET AND NIKOLAY TZVETKOV 



Let us next check the assumption (|2.8j) . We have 

(Lv, v) + (S(ik)v, v) > \v x \ 2 + \v\ 2 + k 2 \d x x v\ 2 - C\v\ 2 , 
where C = p\\Q p ~ 1 \\l°°{r) an d hence using the Fourier transform, we find 



(Lv,v) + (S(ik)v,v) > 



(2VT)- 1 / (e+ k ^ + i-c)\m\ 2 di. 



and since £ 2 +k 2 /£ 2 > 2C for \k\ >C, we get (f!T8|) (more precisely for \k\ > C, £ 2 + k 2 /£ 2 > 
2\k\ > 2C). 

Let us next turn to the assumption on the eigenvalue problem in the context of (|8.5p . 
The resolvent equation reads 

(8.6) au = d x (-d 2 x + 1 - pQ p-1 )u - k 2 d~ l u + F x . 

To prove the existence of the Evans function and (|2.14p , (|2. 16[) , we shall use the criterion 
of section HI Let us define 

d x , if k ^ 0, 
Id, if k = 0. 

then we directly find that R(a — J(L + S(ik)) = Pi (a, k) is a differential operator of order 
4 for k ^ and 3 for k = 0. Consequently, the assumption 14. II is matched with an empty 
second block. 

For k 7^ 0, we have (U3D with 

/ 1 




\ -k 2 - pdliQP- 1 ) -a - 2pd x (QP~ 1 ) 1 - pQ^ 1 j 



R(a, k) 



A(x, £7, k) 






M 


1 








1 


pQ p ~ l 


) 



Thus 



Aocia, k) 



J 



( 1 \ 

10 

1 

\ -k 2 -a 1 

The eigenvalues of A OQ (a, k) are the roots of the polynomial P 
(8.7) P(X) = A 4 - A 2 + aX + A; 2 

and hence are not purely imaginary when Rec > 0, k ^ 0. Moreover, there are two of 
positive real part and two of negative real part. For k = 0, we have 

1 

A(x,a,0) = [ 1 

-a - pd x {Q p ~ l ) l-pQ p " 1 
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and thus 



Aoo(a, 0) 









The characteristic polynomial of A OQ (a, 0) is p(A) = —A 3 + A + a and thus for Re(cr) > 

the eigenvalues of A OCl (a, 0) do not meet the imaginary axis. 

Consequently, the existence of the Evans function follows from Lemma 14.11 

Finally, since the KdV solitary wave is stable (see e.g. [26]), we have D(a, 0) ^ when 

Rec > and hence the assumption (|2.12p is met. Consequently, (|2.14p follows from (|4.2p 
To handle the localized case, we note that when k tends to zero, there is a single root 

A = of (|8.7p on the imaginary axis and hence, there is spectrum of A OQ (a, + ) on the 

imaginary axis. More precisely, for k ~ this root behaves as 



fj,(a, k) 



a 



Consequently, there is only one of the negative real part roots of (|8.7p which goes to zero. 
Since n(a,k) is analytic, we can use the Gap lemma [TT], [T7] to get the continuation of 
the Evans function. Moreover, for k close to zero, we can write the Evans function as 



D(a,k) 



f\ (0,o-, k) 

d 2 x ^(0,a,k) 
dlVx (0, a, k) 



(0,cr,fc) 
d x if2 (0,cr, k) 



<Pi(0,a, k) 
d x ipf(0,a, k) 
d 2 xV >t(0,a,k) 



(p%(0,a,k) 
d x ^(0,a, k) 
c^+(0,a,fc) 
d 3 x ^(0,a,k) 



1,2 



where (ff(x, a, k), i = 1, 2 decay when x goes to ±oo for k ^ 0. When k = 0, ip~ , 
and ff keep this property as ip%(x, a, 0) = c + 0(e~ a l x l), where c / and a > are some 
fixed constants. Note that <p~ (x, a, 0), i = 1, 2 and ^(£, cr, 0) actually solve 



(8.9) 



a^a^-^ + l-pQ^V 



which is the linearized KdV equation about the solitary wave whereas after integration, 
we get that <£>2~(a;, a i 0) sorves 



(8.10) 



au 



d x {-&i + 1 - pQ^u + ca, 



where the source term ca is identified by looking at the value at oo of 

acp+ (x, a, 0) - d x {-d 2 x + 1 - pQ p - 1 )<p+ (x, a, 0) . 

Consequently, using (|8.9p . (|8.10p we can write the forth derivatives of ipf (0,a, 0), i = 1, 2 
as the same linear combinations of lower order derivatives with an additional term ca for 
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Cf2 (0,cr, 0). Therefore, we can perform an operation on the line of the determinant which 
defines the Evans function, to get that 

<Pl(0,a,0) <pZ(0,a,0) ^(0,(7,0) <ft(0,a,0) 

d x (pi(0,a,0) d x <p 2 (0,a,0) d x (pf(0,a,0) d x ip%(Q,a,0) 

d 2 ^(0,a,0) d 2 ^(0,a,0) d 2 <p+(0,a,0) c^+(0,M) " 
co- 

Consequently, we get that |X>(cr, 0) | = \caD(a,0)\ where D(a, 0) is the Evans function 
associated to the linearized KdV equation about the solitary wave. Again, since the KdV 
solitary wave is stable (see e.g. |26j). we also have D(a, 0) does not vanish for Re a > 
and hence, (|2.15p is verified. Finally, (|4,8p is also met in view of f|8.8j) since 

R{a,k)J(ik)S(ik) = d xx {-k 2 d x 2 ) = -k 2 . 

Therefore, (|2.16|) follows from Lemma 14.31 

The assumptions of section 12.3.41 on the existence of a multiplier M s are also matched. 
Indeed, we can use the criterion given by Lemma 15. 11 Let us set 

K s w = r s (x) w 

where r s is a smooth and real valued function. A few computation give 

E s u = ^d x {{pQ p - l ) x d x u) - S -{- ( P Q p - 1 ) x d xx u + M-W 1 )*")) " \\~ d l + d ^r s ]u 

= ({^ + s)( P Q p - 1 ) x + ^{r s ) x ^d xx u + E s u 
where E s is a first order differential operator. Consequently, with the choice 

the properties (|2.17|) . (|2.18|) are verified. Notice that a similar argument can be performed 
each time we deal with a scalar equation, i.e. d = 1 in our general framework. 

The "nonlinear" assumptions in the context of (|8.5|) are also met. In the context of 
(183]) . (f27T9l) becomes 

(8.11) d t u = -u xxx + u x + d x l u yy - d x [{u a + uf - {u a ) p ] + d x G, u(0) = 0. 
To check (j2.20p . we have to estimate 

J d a d x ((w + vf' 1 v) d a v dxdy 

with d a = d^dy 2 , \a\ \ + | «2 1 < s. Therefore we need to study 

J d a d x (w q v r ) d a v dxdy, 0<q<p—l, q + r = p, 
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where w may only be putted in L°° (or some of its derivatives). If at least one of the 
derivatives of d a d x acts on w q then we can use the Gagliardo-Nirenberg-Moser estimates 
to get the needed bound. Therefore it remains to study 

w g d a d x (v r ) d a vdxdy = r I w q d a (v r - 1 d x v) d a vdxdy 



We write 

fld a (v r - 1 d x v)d a v = J w q v r - 1 d x d a vd a v + J w q [d a ,v r - 1 ]d x vd a v 

= ~\f d^v 1 - 1 )^^ 2 dxdy + J w q [d a ,v r - 1 ]d x vd a v 

and hence (|2.20p follows by the Sobolev embedding and the classical tame commutator 
estimate 

II [&>, f]g\\ <C p (\\f\\ k \\g\\ L ~ + ||V/|U» HsHfc-i) , l<|/3|<fc. 

As already used in the general framework the estimate (|2.20p and the fact that JLq and 
JS are skew-symmetric allow to get and M s energy estimate for ()8.11[) . 

To get the well-posedness of ()8.11[) . the procedure is very classical and there are several 
possibilities to achieve this conclusion. One possibility is to consider a regularized version 
of (|8.1ip . for example 

(8.12) d t u £ + eA 2 d t u e = -ul^ + ul + d^vfyy- d x [{u a + u e f - (u a ) p ] +d x G, u £ (0) = 0, 

where e > and A = d 2 + d 2 . Thanks to the regularization (which is by no means 
canonical), we may solve (18.12j) for a finite time, independent of e > 0, by means of the 
Picard iteration applied to the associated integral equation. We next wish to pass to the 
limit e — > + in u £ . For that purpose, we need to establish an apriori bound on ||u e (t, -)\\h s 
independent of e. These bounds follow from the fact that the well-chosen perturbation 
enjoy the same H s estimate as the one formally obtained for (18. lip . Then we pass to the 
limit e — > + thanks to a compactness argument. This establishes the local well-posedness 
of flgXEp . 

Finally, let us notice that the sufficient condition of Lemma 13.11 for the existence of an 
unstable mode applies. Indeed, we have 

M k = d x Ld x - k 2 Id, 

therefore, it suffices to show that the self adjoint operator d x Ld x on L 2 (R), with domain 
H 4 (1H), has a unique positive eigenvalue. Note that, thanks to Weyl's theorem the essential 
spectrum of d x Ld x is ] — oo, 0] . Therefore on [0, oo] the spectrum of d x Ld x can only contains 
eigenvalues of finite multiplicity and hence for k > 0, M k is Fredholm with zero index. 
Since L has a unique negative eigenvalue and the remainder of its spectrum is included in 
[0, oo], we obtain that, by analyzing the corresponding quadratic forms, the operator d x Ld x 
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cannot have more than one positive direction, i.e. u such that (u, d x Ld x u) > 0. Let us 
finally show that a positive direction indeed exists. Let us denote by u_i the L 2 normalized 
eigenvector of L with corresponding to the negative eigenvalue — \i. Let ip n G H W {W) be 
a sequence such that d x ip n converges to U-\ in ff 10 (M). Then (d x Ld x ip n , cp n ) converges 
to fi > 0. Therefore there exist a positive direction of d x Ld x which shows that d x Ld x 
has a positive eigenvalue (recall that on R + the spectrum of d x Ld x can only contains 
eigenvalues) . 

Finally, for k$ the unique positive eigenvalue of d x Ld x , we have 

([^M fc ] fc=fco <^, tpj = -2k ± 

and hence (j3.1|) is verifed. Thus Lemma 13. II applies in the context of (18. 5]) . 

Therefore we can apply our general theory and obtain that Q is (orbitally) unstable as 
a solution of (|8.5p (posed on KxT„ with a suitable a or M. 2 ) thanks to our general results. 
We have the following statement. 

Theorem 8.1. For every s > 0, there exists r/ > such that for every 5 > there 
exists Uq and a time T s ~ | log<5| such that \\uq — Q\\h»(m?) < ^ an( ^ ^ e generalized KP-I 
equation $8.5\) with data Uq is locally well-posed on [0, T s ]. Moreover, if we denote by u s (t), 
t E [0,T a ], the corresponding solution, then u s (t) — Q G H S (R 2 ) for every t e [0,T 5 ] and 

mf \\u 5 (T 5 ) -v\\ L 2 (R 2) > n, 

where T is the space of L 2 (R) functions independent of y. 

A similar statement may be done for periodic in y solutions with a suitable period 
depending on the transverse frequency of the unstable mode (see Theorem [1] above). 

Let us recall (see [2H [28]) that for p = 3, 4 the generalized KP-I equation, posed on 
M 2 has local smooth solutions blowing up in finite time, i.e. another (stronger) type of 
instability exists in these cases. This is in sharp contrast with the case p = 2, i.e. the 
"usual" KP-I equation when global smooth solutions exist both in the case of data periodic 
in y (see [2]) or localised with respect to Q (or zero), see [25]. 

8.2. The nonlinear Schrodinger equation. The Id model is 

(id t + d 2 x )u+ \u\ 2 u = 0, u:M.^C. 

This equation has a solitary wave solution of the form u(t, x) = e lt Q{x) with Q smooth 
with exponential decay. More precisely Q(x) = \/2(ch(x))~ 1 . Then after changing u in 
e*'u, Q becomes a stationary solution of 

(8.13) (idt + d 2 x )u - u + \u\ 2 u = 0, u : R -> C. 
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By writing u = u\ + 1U2 with real valued u\,U2, we obtain that U = («i,«2)* solves the 
equation 



(8.14) d t U=l _° : I 

which fits in our framework with 
d = 2, J 



(-^ + 1)(/ + VF([/) , 



1 
-1 



£0 



-a 2 + Id. 



The solution Q of (|8.13p is orbitally stable (see [6]). This implies the orbital stability of 
(Q,0) as a solution of (|8. 14j) . The operator L in the context of (|8.14p is given by 



L 



-dl 



Id 




3Q 2 





-d 2 x + Id - Q 2 



The spectral condition (|2. 1 1 j) on L is satisfied since — d 2 + Id — 3Q 2 has exactly two simple 
eigenvalues —3 and with corresponding eigenvectors Q 2 and Q' and continuous spectrum 
[1, 00 [ while — d 2 + Id — Q 2 has one simple eigenvalue with corresponding eigenfunction 
Q and continuous spectrum [l,oo[ (see e.g. [30| IBTj). 

The transversely perturbed model is the 2D NLS equation that we can write 



3.15) d t U 



1 
-1 



-d 2 + l)U + VF(U)-d 2 U ), F(U) 



\(u\+u 2 ) 2 , 



i.e. S{d y ) = —d 2 and S(ik) = k 2 Id. The assumptions of sections I2.2.H [2T2T21 are easy to 



check. Since Q is bounded, assumption (|2.8|) is also trivially satisfied. 

Let us next turn to the assumption on the key eigenvalue problem in the context of 
()8.15j) . Again, we shall use the criteria of section HI This is very simple in this case, 
since a — J(L + S(ik)) is already a differential operator. Consequently, we can take 
R(a,k) = Id. If we introduce V = (u\,U2,d x ui,d x U2) t £ C 4 , F = (0,0,^2,-^1)* we can 
rewrite the resolvent equation as V x = A(x, a, k)V + F, where for all k, 

( 1 \ 

1 

k 2 + l-3Q 2 a 
—a 



A(x, a, k) 



V 



Thus 



A OQ (a,k) 






k 2 + l 
—a 



k 2 + l-Q z 0/ 



1 \ 

1 

a 

k 2 + l J 
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and we see that A and are analytic in (<r, k). We then define D(a, k) as the Wronskian 
associated to A(x, a, k). Thus D(a, 0) = D(a, 0) in the case of the NLS. The eigenvalues 
of Aqq (<t, fc) are the roots of the polynomial P 

(8.16) P(A) = (A 2 - k 2 - l) 2 + cj 2 . 

Therefore, in the context of (I8,15p . for every k £ M the spectrum of ^4oo(<7, k) does not 
meet the imaginary axis. Thus the assumption (|4.8|) is obviously satisfied. Moreover, since 
D(a, 0) = -D(o", 0), (|2.12p and (|2.15p are met because of the ID stability of the solitary 
wave. 

Since J is a zero order operator and Lq has the required form, we can use Corollary 15.21 
to get the existence of a multiplier. 

The nonlinear assumptions in the context of (|8.13p is satisfied thanks to the standard 
well-posedness argument for the 2D NLS equation 

Moreover, since here J is of order zero, the estimate (|2.20p follows by the standard 
Gagliardo-Nirenberg-Moser inequalities. 

Finally, the sufficient condition given by Lemma 13.11 for the existence of an unstable 
mode applies. Indeed, as for the KP equation, we have 

M k = JLJ - k 2 Id 



since 

jrr- ( "5 2 +Id-Q 2 

JLJ --[ o -d 2 x + Id - 3Q 2 

which have a unique positive eigenvalue. The non-degeneracy condition (|3,ip is also obvi- 
ously verified. 

Therefore, our general theory applies and we can state the following results. 

Theorem 8.2. For every s > 0, there exists r/ > such that for every 5 > there exists 
Uq and a time T s ~ | log <5| such that \\uq — Q\\h«(m. 2 ) < ^ an d the two dimensional NLS 
equation 

(id t + dl + d 2 )u + \u\ 2 u = 0, u : R 2 -> C 

with data Uq is locally well-posed on [0, T 5 ]. If we denote by u s (t), t £ [0,7^], the corre- 
sponding solution, then we have u s (t) — Q £ H S (M. 2 ), Vi £ [0, T s ] and 

inf \\u s {T s ) - «|| L 2 (K 2) > rj, 
where T is the space o/L 2 (R) functions independent of y. 



A similar statement may be done for periodic in y solutions with a suitable period 
depending on the transverse frequency of the unstable mode (see Theorem [1] above). 
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8.3. The Boussinesq equation. Consider the Id Boussinesq equation 
(8.17) u tt + (u xx + u 2 - u) xx = 0, 

This equation has a traveling wave solution (see [3]) of the form 

u(t, x) = q(x -ct)=q£ H°°(R; R 2 ), \c\ < 1, c / 0. 
In addition q has an exponential decay at infinity. Note that we have 



q(x) = (1 - c 2 ) Q KdV (Vl-c 2 x) 

where Q KdV is the solitary wave with unit speed of the KdV equation given by (|8.2p (for 
p = 2). Moreover for |c| S]l/2, 1[ this traveling wave is orbitally stable (see [1]). 
At first, we shall rewrite (|8.17|) as a first order equation. Let us define 

Bu = —u xx + u 

and B a as the Fourier multiplier with symbol (|£| 2 + l) a . Note that B a is a symmetric 
operator. By using B, we rewrite (|8.17p as 

ut = d x B^v, v t = d x B~2 (^Bu — u 2 

Changing x into x — ct, we get 

(8.18) u t = d x B~UBv + cB^u), v t = d x B~\( Bu + cB^v - u 2 



With this change of frame, Q(x) = (q(x),—cB ^q{x)) is a stationary solution of (|8.18p . 
By setting U = (u,v) 1 , 



i \ / „ „ i 



^ d x B~2 J' \ cB* B J ' V 

we can write (|8.18p under the form (|2,ip . We easily check that the assumptions of section 
12.11 are matched. Note that by Bessel identity, we have 



(l u, u) = (2VT)- 1 / ((i + \t\ 2 )\m\ 2 + (i + \t\ 2 )\m\ 2 + 2c (Vi+wm m) <% 



and hence, (|2.2p is verified for c < 1. Moreover, an important remark is that J is here a 
bounded operator, in contrast with the formulation used by [3]. 

Next, let us check (|2.3|) . The operator L is defined by L = Lq + i?, where 



i? 



-2g 




The spectral condition on L is again satisfied thanks to the Sturm-Liouville theory and 
is proven for 1/2 < |c| < 1 in [3] in order to prove the nonlinear stability of the solitary 
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wave. Note that the formulation (|8.18p that we use is equivalent to the one used by Bona 
and Sachs in [4]. 

The transversally perturbed model is 



(8.19) u tt + (u xx + u 2 -u- d x 2 d 2 u) xx = 0. 

The equation f|8. 19|) has been derived in [15] as a model for interacting shallow water 
waves. Again, to rewrite this equation as a first order system, we introduce 

Bw- - ' - 1 «~ 2 " 

We write (ETtTfl) as 

(8.20) u t = d x B?v, vt = d x B~ 
and hence going into the moving frame, we find 



-W XX + W + 8 X Wyy. 



u t 



d x B-2[Bv + cB2'. 



d x B 2 [Bu + cB?v - u' 



Consequently, we get a system under the form (12. 4j) with 
Therefore the Id operators J(ik) and S(ik) are defined as 



S(dy) 



d x 2 dl 
c{B^-B l 2] 



c{Bh - 
d x 2 d 2 



-k 2 d x 2 

c(J3(tfc)3 - Bs) 



with 

B(ik)w = —w xx + w — k 2 d~ 2 . 
Note that J(ifc) is a bounded operator on L 2 . Indeed, its symbol is given by 



c(B(tfc)3 - si) 







We can easily check that the assumptions of section [2. 2. II are verified. Since J is bounded 
on L 2 , the estimate (|2.6p and (|2.5j) are obvious. 

We also easily check the assumptions of section 12.2.21 The first three assumptions can 
be verified by computation. Moreover, we notice that there exist positive constants Co, Co 
such that 







3.21) 



cok^d^Ul 2 < (S(ik)U,U) < Cok^d^U] 2 . 
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Indeed by using the Fourier transform, we have 

k 2 

(S(zk)U,U) = (2VT)" 1 / (^(|fi(0| 2 +|0(O| a )+2c -fi r*(0 

(i + e 2 + f) 5 + (i + e 2 ) 5 y 

and hence (|8.2ip follows since c < 1. Thanks to (|8.2ip . we also find that (|2.7p is verified 
with C{k) = k. 

Next, to get (|2.8p . we use again the Fourier transform to write 

(LU, U) + (S(ik)U, U) > d / (1 - c) (1 + £ 2 + |J - C) |E/(0| 2 #, 

where Ci « |Q|l°°- Consequently, we get (|2.8p as for the KP equation. 

The assumptions of section 12.3.41 on the existence of suitable multipliers follows again 
from Corollary 15.21 Indeed, J{ik) is a zero order operator and Lq = —d 2 + L with L a 
first order operator. 

Let us turn to the study of the resolvent equation (|2,13p . We first notice that 



aid- J{ik){L + S{ik)) 



a — cd x —d x B{ik) 
^ -d x B(ik)~l (B(ik) -2q) a- cd x 

Consequently, by using again section [H we can set 



R(a,k)=( a - Q cd * 9 - B ^ k ) h \ 



to get (gJJ) with 

Pi(cr, k)u = d x u — d 2 u + k 2 u + 2d 2 (qu) + (a — cd x ) 2 u, 
E(a, k) = a — cd x , 

P 2 (a,k)u = -d x B(iky^(B(ik)u - 2qu) = -d x B(ik)^u - 2d x B(ik)~^ (qu). 

Consequently, Pi is a fourth order differential operator analytic in (k, a) for every k, E 
is invertible for Re a > and P 2 is a second order operator with domain H 2 . Indeed, 
d x B(ik)~z is a bounded operator on L 2 and we have the estimate 

2^\\d x B{ik)^u\\ 2 = [ e 2 (c 2 + i + ^)in(oi 2 ^< / (i+e + k 2 )\m\ 2 dt, 

which is uniform for A; in a vicinity of zero. 

One can rewrite (|4.3p in the context of the Boussinesq equation as a first order system 
(|4.4p with A(x,a, k) S M^C) for every k. The assumption (|4.6p is met since q decays 
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exponentially fast to zero at infinity. Moreover, the eigenvalues A of A OQ (a, k) are the roots 
of the polynomial P defined as 

P(A) = A 4 - (1 - c 2 )A 2 - 2caX + k 2 + a 2 . 

Suppose that P has a root of the form A = iu with /ieR. Then by separating the real 
and the imaginary part of P(ifj>), we get the relations 

// + (1 - c 2 V 2 + 2ca 2 u + a\ - of + k 2 = 0, -2co\u + 2a^ 2 = 0, 

where a = o\ + 102, 01, 02 £ M. Therefore, since Re(er) = cri 7^ 0, we have that \x = 02 /c 
(recall that we are interested for the values of c such that 1/2 < |c| < 1). By substituting 
the value of [i in the first equation, we get 

(8.22) 4 + 4 + ^ + ^ = 0. 

But since in the last equation for 02, if 02 is real, all the terms are non-negative and a\ > 0, 
there is no real root for every k G BL Therefore for every k G K. and Re(a) 7^ the equation 
-P(A) = has no root on the imaginary axis. Since for k = there is no complication 
coming from the emergence of a root on the imaginary axis, we are in the same situation 
as for the nonlinear Schrodinger equation. The assumption (I2.12|) (and hence also (|2.15p ) 
is met since the solitary wave q is stable as a solution of the ID Boussinesq equation for 
1/2 < I c| < 1 as shown in [3] , we get (I2.14|) , (I2.16D from Lemma 14.21 and Lemma 14.31 

The "nonlinear" assumptions of section 12.41 are also met. Indeed, the local well- 
posedness of the 2d Boussinesq equation which is semi-linear can be obtained by standard 
techniques. Moreover, since J is a bounded operator on H s , the assumption (|2.20p follows 
readily from the Gagliardo-Nirenberg-Moser inequality. 

Remark 8.1. Let us observe that if we consider transverse perturbation with the opposite 
sign that even the problem defining the free evolution is ill-posed in Sobolev spaces. Thus 
in the context of the Boussinesq equation the analogue of the KP-II equation is not a 
"good" model in Sobolev spaces. Indeed consider the linear problem 

(8.23) u tt + {u xx -u + d~ 2 d 2 u) xx = 0, 
Using the Fourier transform, we obtain that u solves 

u tt + {t + f-k 2 )u = 0, 

and hence, one can find growing modes e A *u(£, k) if 

P(A) = A 2 + £ 4 + f - k 2 = . 

Since one can find roots of P with arbitrary large real parts and arbitrary sign this implies 
the ill-posedness of (|8.23p . A similar phenomenon occurs for the equation 

(8.24) u tt + {~u xx -u- d~ 2 d 2 ) xx = 0, 
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where the sign is changed in front of the dispersion term. In view of this discussion, it 
becomes reasonable to study (|8.23p or (|8.24p in analytic spaces. 

We can also use Lemma [3J] to get an unstable eigenmode. Indeed, we have 
M k ~- 



d X x cd xx B(ik) 2 

^ cd xx B(ik)~h d x B(ik)-^(B(ik) - 2q)d x B{ik)~^ 

Consequently (u, v) 1 G L 2 (R; M 2 ) is in the kernel of M k if and only if 



u = -cB(ik)~*v, d x B{ik)~i [ - c 2 + B(ik) - 2q\d x B(ik)~iv = 0. 
Next, we notice that 

d x B(ik)-^ ( - c 2 + B(ik) - 2q\d x B(ik)~* = 

= B(ik)-*(d x ( - d 2 x + (1 - c 2 ) - 2q)d x - k 2 )B(iky^ = B{ik)~^m k B{ik)~^ . 



Notice that is the operator JLJ + JSJ which appears in the study of the stability of 
the solitary wave with speed 1 — c 2 of the KP-I equation. Thus as in the analysis for the 
KP-I equation, we can show that has a one-dimensional non trivial kernel for some 
ko 0. This implies that M ko also has a non-trivial kernel generated by 

cp = (-ctp,B(ik )2tp), 

tp being nontrivial and such that mk ip = 0. Moreover, we can deduce that M ko is Fredholm 
index from the fact that mk is Fredholm index 0. Let us check the non-degeneracy 
condition (13. ip . Using the identity 



(8.25) 4r B{ik)~^ = k d~ 2 B(ik y 

dk /k=k 

we obtain that 

(8.26) {\^-M k } i p,i P )=-2k \iP\ 2 ^0 

x ldk ik=k ' 

since ko ^ 0. More precisely M k = + Mf with 

d xx 
d x B(ik)~^ (B(ik) - 2q)d x B{ik)- L 2 



Ml 



and 



M 2 = cd xx B(ik) 2 

k I cd xx B(ik)-h 
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A use of (jQ5l) gives 

( \^rMi] if, <p) = -2<?k Q {B(ik Q y 1 ^ 
idk J k=k 

and (using that mk tp = 0) 

( l-^Ml] if, if) = 2c 2 k (B(ik )- 1 i>, V) - 2fc |Vf • 

Thus the identity (|8.26|) indeed holds true. This allows to use Lemma 13.11 to get the 
existence of an unstable eigenmode. 

Consequently, we can apply our general theory to get the following statement. 

Theorem 8.3. Consider the equation (|8.20p for \c\ E (1/2,2). For every s > 0, there 
exists T] > such that for every 5 > there exists Uq and a time T s ~ | log <5| such that 
\\ u o ~ Q\\h s (r 2 ) < ^ an d th e solution u s (t) of (|8.20p with data Uq is defined on [0,T S ], with 
u s (t) — Q G H S (M. 2 ), yt G [0, T s ] and moreover satisfies the estimate 

in f h S (T S ) -v\\tfrga\ > V, 

where T is the space o/L 2 (M) functions independent of y. 

A similar statement may be done for periodic in y solutions with a suitable period 
depending on the transverse frequency of the unstable mode (see Theorem [T] above) . 

8.4. The Zakharov-Kuznetsov equation. The Zakharov-Kuznetsov equation 

(8.27) u t + u xxx + u xyy + uu x = 

is derived in |32j to describe the propagation of nonlinear ionic-sonic waves in plasma 
magnetic field. Equation (18. 27ft is a two dimensional generalization of the KdV equation 
which fits into our general framework with d = 1. Indeed, if we denote by Q the suitable 
speed one KdV solitary wave, then Q is a stationary solution of 

(8.28) u t -u x + u xxx + u xyy + uu x = 0. 
We can write (|8.28p as 

with 



u t = J{L Q + VF{u)+S{d y ))u 



J = d x , L = -^ + Id, F(u) = - — , s(d y ) = -d z y 



ii 
"6 

Assumptions of section 12.11 are still verified since as for the KP-I equation, the Id model 
is the KdV equation. Assumptions 12.2. H 12.2.21 (|2-8j) are easy to check. 
The operator 

au — J(L + S(ik))u = au — u x + u xxx — k 2 u x + 2(Qu) x 
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is already a differential operator and hence we can readily use section 0] with R(a, k) = Id. 
In particular, we have for every k 

I 10 

A(x,a,k) =10 1 

\ -<J-2Q X (l + k 2 )-2Q 

This allows to find that the eigenvalues of (er, A;) are the roots of the polynomial 

P(A) = A 3 - (1 + k 2 )\ + a. 

Since, for £ G E, we have Re P(i£) = Re a, we get that for Re a / there is no eigenvalue 
of A OQ (a, k) on the imaginary axis. As in the case of NLS, the assumptions of section 1431 
are obviously verified since we are in a situation where A(x, a, k) is analytic for every k 
and where there is no eigenvalue of A^ on the imaginary axis even for k = 0. Moreover, 
(|2.12p (and hence also (12. 15ft ) are verified thanks to the stability of the KdV solitary wave. 
Consequently, the assumptions of sections 12.3.21 and 12.3.31 follow from Lemma 14.21 and 
Lemma 14.31 

To check assumption 12.3.41 we use again Lemma 15.11 By taking 

2 

K s u = — (1 + 2s)Qu, 
3 

we find that E s is a first order operator and hence the assumption of existence of multiplier 
of section [2.3.41 follows from Lemma 15. 11 

The assumptions of section [23] i.e about the local well posedness of the nonlinear equa- 
tion are again verified by standard arguments. Note that assumption (|2.20p was already 
checked in the study of the KP equation. 

Finally, we note that it does not seem possible to use the simple criterion of Lemma [3.11 
to prove the existence of an unstable eigenmode. Indeed, we have 

M k = d x {L - k 2 )d x , L = -u xx + u - 2Q. 

It is easy to prove that there exists ko such that Mk Q has a non-trivial kernel. Nevertheless, 
here Mk is not a Fredholm operator with index zero. Fortunately, the existence of unstable 
modes was obtained in [5] by using more sophisticated arguments (i.e. the multisymplectic 
formulation of the equation). Consequently, we have the following result. 

Theorem 8.4. Consider the equation (|8.28|) . For every s > 0, there exists 77 > such 
that for every 5 > there exists and a time T s ~ | log <5| such that \\uq — Q\\h s (m?) < ^ 
and the solution u 5 (t) of (^28|) with data u 5 Q is defined on [0,T 5 ] with u 5 (t)-Q £ H S (R 2 ), 
Vi G [0, T s ] and moreover satisfies the estimate 

in f \\ uS {T S ) ~ HliW) > V, 

where T is the space of L 2 (R) functions independent of y. 
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A similar statement may be done for periodic in y solutions with a suitable period 
depending on the transverse frequency of the unstable mode (see Theorem [T] above). 

8.5. KP-BBM. Consider the generalized BBM equation 

(8.29) u t - u txx + u x + d x (u p ) = 
and the 2d generalization of KP type 

(8.30) u t - u txx + u x + d x (u p ) - d~ x u yy = 0. 

For c > 1 there is a solitary wave solution of (|8.29j) of the form u(t, x) = Q{x — ct). Again, 
we note that 



1 



Q( x ) = (c-l)—iQ KdV (^l--x). 
Then Q(x) is a stationary solution of the equation 
(8.31) u t - (c - l)u x - utxx + cuxxx + d x (u p ) - d~ x u yy = 0. 

Equation (|8,3ip may be written under the form 

d t u = J(L + VF(u) + S(d y ))u, 

where 

J = {l-d 2 x )- l d x , L = -cd xx +{c-l)Id, F(u) = -—^ J u p+1 , S{ik)u = -k 2 d~ 2 . 

The corresponding operator L is 

L(u) = —cu xx + (c — l)u — pQ p ~ l u . 

Again, it is very easy to check the assumptions of sections \2.1\ 12.2.14 12.2.21 12.81 To ensure 
that (|2.1ip and (|2.12p are verified, we restrict ourself to p < 4, in this case all the waves 
for c > 1 are stable in the ID model which is the BBM equation [26], [3]. 

Note that we are in a semilinear situation since J is a zero order operator (and even 
better). Consequently, the assumption of existence of multiplier of section f2.3.4l is veri- 
fied thanks to Corollary 15.21 (recall that for scalar problems it is straightforward). The 
assumption 12.201 is met thanks to the Gagliardo-Nirenberg-Moser inequality. Again the 
local well-posedness assumed in section [23] can be proven by standard methods. 

To check (I2.14p . (I2.16p . we can use section [H Since 

au - J(L + S{ik))u = au-{l- d 2 x )- 1 d x { - cd xx u + (c - l)u - P Q p ~ l u - k 2 d~ 2 u) , 
we set 

(u-d 2 )d x , iik^o, 



R(*,k) = {^:< 



I if = o. 
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Then we directly find that R(a, k) — J{L + S(ik))) = Pi(cr, k) is a differential operator of 
order 4 for k 7^ and 3 for k = 0. Consequently, the assumption of section |4~T1 is matched 
with an empty second block. 



For k 7^ 0, we have (|44j) with 

/ 

A(x, a, k) = c _1 



V 



-k 2 






pd 2 x (Q p ~ 



c 






c 




\ 


c 



Thus 



Ax>(cr, fe) 



V 







-k 2 



c 



— <T 





c 


c- 1 



\ 




The eigenvalues of A^a, k) are the roots of the polynomial P 
(8.32) P(A) = cA 4 - crA 3 - (c - 1)A 2 + cjA + /c 2 

and hence are not purely imaginary when Recr > 0. Moreover, there are two of positive 
real part and two of negative real part. For k = 0, we have 

c 
A(x,a,0) = c 1 [ 



and thus 



^oo(o",0) = c" 






—a 




The characteristic polynomial of A^a, 0) is p(X) = cX — aX — (c — 1)A + a and thus for 
Re(cr) > the eigenvalues of A OQ (a, 0) do not meet the imaginary axis. This allows to use 
Lemma 14.11 to get the existence of the Evans function. Finally, since the BBM solitary 
wave is stable (see e.g. [26], [3]) for p < 4, c > 1, we have D(a, 0) / when Recr > and 
hence the assumption (|2.12p is met. Consequently, (|2.14p follows from (|4.2p 

To handle the localized case, we note that when k, tends to zero, there is a single root 
A = of (|8.7p on the imaginary axis and hence, there is spectrum of A OQ (a, + ) on the 
imaginary axis. More precisely, for k ~ this root behaves as 

k 2 

(8.33) MO,*0 • 

a 

Consequently, there is only one of the negative real part roots of (|8.32p which goes to 
zero. Since /j,(a, k) is analytic, we can use the Gap lemma [11], [17] to get the continuation 
of the Evans function. Moreover, by using the same method as in the study of the gKP 
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equation, we can also write the Evans function as \D{a, 0)| = \caD(a, 0)| where D(a, 0) 
is the Evans function associated to the linearized BBM equation about the solitary wave. 
Again, since the BBM solitary wave is stable we also have that D(a, 0) does not vanish for 
Re a > and hence, (I2.15|) is verified. Note that, (|4.8|) is also met in view of (18.330 since 

R(a,k)J(ik)S(ik) = d xx (-k 2 d x 2 ) = -k 2 . 

Therefore, (|2,16p follows from Lemma 14.31 

Finally, as for the gKP equation, the existence of an unstable eigenmode follows from 
Lemma |3. 11 Indeed, we can write M k under the form 

M k = (l-d 2 )- l m k (l-d 2 x )-\ 

where 

m k u = cd x (d x (-d xx + ^^Id + ^QV- l Id)d x - k 2 . 

V c c / 

Again, the existence of a nontrivial kernel for m k comes from the study of the KP equation 
and one can deduce that M k is Fredholm from the fact that m k is Fredholm. 
Therefore, we can state the following result. 

Theorem 8.5. Consider the equation (|8.3ip for c > 1 and p < 4. For every s > 0, there 
exists rj > such that for every 5 > there exists Uq and a time T 5 ~ | log <5| such that 
\\uq — Q\\h s (r 2 ) < ^ an d th- e solution u s (t) of (|8,3ip with data Uq is defined on [0,T S ] with 
u s (t) — Q G H S (M. 2 ), yt G [0, T s ] and moreover satisfies the estimate 

in f \\ uS {T S ) ~ w||l 2 (m 2 ) > V, 

where T is the space of L 2 (R) functions independent of y. 

A similar statement may be done for periodic in y solutions with a suitable period 
depending on the transverse frequency of the unstable mode (see Theorem [T] above) . 

Let us point out that the KP-BBM model considered in this section is not the relevant 
one from modelling view point (see [22]), the relevant one being 

(8.34) m - (c - l)u x - u txx + cu xxx + d x (u p ) + d~ l u yy = 0. 

Equation (|8.34p does not fit in the framework considered in this paper and it is possible 
that the KdV soliton is in fact stable as a solution of (|8.34p . Nevertheless our KP-BBM 
model seems interesting for the following reason. 

8.6. Final remark. Let observe that in the case p = 2 the equation (|8.31j) is globally well- 
posed for data close to Q. We have therefore nonlinear instability in the context of global 
well-posedness. Therefore this type of phenomena already encountered in the context of 
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the KP-I equation is not only restricted to integrable models as the KP-I equation. Let 
us briefly explain how we prove the global well-posedness for 

ut- (c- l)u x - u txx + cu xxx + d x (u 2 ) - d~ l u yy = 

with initial data 

u(0,x,y) = Q(x) + v (x,y), 

where vo is localized both in x,y. More precisely, we suppose that vq £ H S (M. 2 ) with s 
large enough. If we set u = Q + v then we have that v solves the problem 

(8.35) v t -(c- l)v x - v txx + cv xxx + d x (v 2 ) + d x (Qv) - d~ l v yy = 0, v(0, x, y) = v . 

In the case Q = the above equation is shown to be globally well-posed in [29] . In 
the case of a Q which is bounded together with its derivatives one needs to combine the 
argument of [29j with the following control on the flow of (j8.35j) . Multiplying (I8.35h by v 
and integrating over M 2 yields 

^ t (\\v(t,-)\\h + \\d x v(t,-)\\ 2 L2 ) = -2 J d x (Qv)v = -J Q'v 2 . 

A use of the Gronwall lemma provides the control 

(8.36) |K*, -)\\ L 2 + \\d x v(t, -)|| L2 < (llvoll^ + ||5 a; t;o|| i2 ) e c(1+l * l) . 

The local analysis of [29] shows that in the case Q = the problem (|8.35p is locally well- 
posed for data such that ||uo||l 2 + II^x^oIIl 2 < °°- bi order to include the term d x (Qv) in 
the local analysis of [29] one needs to evaluate the quantity 

(8.37) ||(1 - dl)- l d x (Qv)\\ L , + , L2 + ||(1 - d 2 x )- l d 2 x {Qv)\\ L ^ L2 

for some e > 0. The unessential loss e (compared to the natural L\, coming from the 
Duhamel formula) is related to the fact that the well-posedness in [29] is established in 
Bourgain spaces and the non-linearity in a Bourgain's norm of type X^ b ~ l , b > 1/2 close 
to 1/2 can be estimated by the non-linearity if L}^~ e H s with e > close to zero. But 
the quantity (|8.37[) can be easily estimated in terms IMIl^pl 2 + II^Hl^l 2 (and even only 
IMIl°°l 2 ) which shows that the term d x (Qv) can be incorporated in the local analysis of 
|29| which in turn thanks to the control (|8.36p implies that in the case p = 2 the equation 
(|8.31j) is globally well-posed for data which is a localized perturbation of Q. 
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